You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "tolbertam (via GitHub)" <gi...@apache.org> on 2025/02/14 01:29:54 UTC

[PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

tolbertam opened a new pull request, #3899:
URL: https://github.com/apache/cassandra/pull/3899

   As part of a broader effort to decouple java driver code from the server code, this moves sstableloader to its own tools directory.
   
   As sstableloader is also used as a library (CASSANDRA-10637) also added a new artifact 'cassandra-sstableloader' that will get deployed to maven along with 'cassandra-all'.  While I expect this is likely a niche use case, this will allow users to continue using BulkExport as a library.
   
   Also updates IDE project files and circleci to utilize new sstableloader specific targets.
   
   For CASSANDRA-20328
   
   ----
   
   This is not currently ready as it needs a lot of testing.  Just creating it ahead of time so I can self review.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2857881495

   `.jenkins/Jenkinsfile` needs editing too.
   
   add the test type here: https://github.com/apache/cassandra/blob/trunk/.jenkins/Jenkinsfile#L166
    (i assume we only need a "small" size)
   
   and add it to all test profiles: https://github.com/apache/cassandra/blob/trunk/.jenkins/Jenkinsfile#L129-L132


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998556216


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   @tolbertam 
   
   Aha I see, because it is all moved under tools/bin/sstableloader. Interesting. That way we might get rid of _everything_ above, no? Just check the script existence and then call it with all the args.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998547027


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   @tolbertam why have you removed all of this?
   
   ```
   if [ -z "$CLASSPATH" ]; then
       echo "You must set the CLASSPATH var" >&2
       exit 1
   fi
   
   if [ "x$MAX_HEAP_SIZE" = "x" ]; then
       MAX_HEAP_SIZE="256M"
   fi
   ```
   edit:
   
   @tolbertam 
   
   Aha I see, because it is all moved under tools/bin/sstableloader. Interesting. That way we might get rid of _everything_ above, no? Just check the script existence and then call it with all the args.



##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   @tolbertam 
   
   Aha I see, because it is all moved under tools/bin/sstableloader. Interesting. That way we might get rid of _everything_ above, no? Just check the script existence and then call it with all the args.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998967647


##########
tools/sstableloader/src/org/apache/cassandra/tools/LoaderOptions.java:
##########
@@ -79,7 +78,6 @@ public class LoaderOptions
      * provide options instead of using these constant fields.
      * @deprecated Use {@code throttle-mib} instead
      */
-    /** @deprecated See CASSANDRA-17677 */

Review Comment:
   I think I dropped it because of the double javadoc.  I think I should just merge it into the doc above, e.g.:
   
   ```
   @deprecated Use {@code throttle-mib} instead (see CASSANDRA-17677)
   ```
   
   will make that change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2030982166


##########
ide/nbproject/project.xml:
##########
@@ -35,6 +35,12 @@
                     <location>${project.dir}/tools/fqltool/src</location>
                     <encoding>UTF-8</encoding>
                 </source-folder>
+                <source-folder>

Review Comment:
   @michaelsembwever todo



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041162185


##########
build.xml:
##########
@@ -664,6 +666,88 @@
         </testmacro>
     </target>
 
+    <!--
+        sstableloader build file
+        -->
+    <property name="sstableloader.build.src" value="${basedir}/tools/sstableloader/src" />
+    <property name="sstableloader.test.src" value="${basedir}/tools/sstableloader/test/unit" />
+    <property name="sstableloader.build.classes" value="${build.classes}/sstableloader" />
+    <property name="sstableloader.test.classes" value="${build.dir}/test/sstableloader-classes" />
+    <property name="sstableloader.manifest" value="${sstableloader.build.classes}/MANIFEST.MF" />
+
+    <target name="sstableloader-build-test" depends="sstableloader-build" description="Compile sstableloader tests">
+        <javac debug="true" debuglevel="${debuglevel}" destdir="${sstableloader.test.classes}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               includeantruntime="false" encoding="utf-8">
+            <classpath>
+                <path refid="cassandra.classpath.test"/>
+                <pathelement location="${sstableloader.build.classes}" />
+            </classpath>
+            <src path="${sstableloader.test.src}"/>
+        </javac>
+    </target>
+
+    <target name="sstableloader-build" depends="build" description="build sstableloader">
+        <antcall target="_sstableloader_build"/>
+    </target>
+
+    <target name="_sstableloader_build">
+        <mkdir dir="${sstableloader.build.classes}" />
+        <javac compiler="modern" debug="true" debuglevel="${debuglevel}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               encoding="utf-8" destdir="${sstableloader.build.classes}" includeantruntime="true">
+            <src path="${sstableloader.build.src}" />
+            <classpath>
+                <path refid="cassandra.classpath" />
+            </classpath>
+        </javac>
+    </target>
+
+    <target name="sstableloader-test" depends="sstableloader-build-test, maybe-build-test" description="Runs sstableloader tests">
+        <testmacro inputdir="${sstableloader.test.src}"
+                       timeout="${test.timeout}">
+        </testmacro>
+    </target>
+
+    <target name="sstableloader-jar"
+            depends="sstableloader-build"
+            description="Assemble Cassandra sstableloader JAR files">
+
+        <!-- sstableloader jar -->
+        <manifest file="${sstableloader.manifest}">
+            <attribute name="Built-By" value="Apache Cassandra Project"/>
+            <attribute name="Main-Class" value="org.apache.cassandra.tools.BulkLoader"/>
+        </manifest>
+        <mkdir dir="${sstableloader.build.classes}/META-INF" />
+        <mkdir dir="${build.dir}/tools/lib/" />
+        <jar destfile="${build.dir}/tools/lib/sstableloader.jar" manifest="${sstableloader.manifest}">
+            <fileset dir="${sstableloader.build.classes}"/>
+        </jar>
+    </target>
+
+    <target name="sstableloader-javadoc" depends="build" description="Create javadoc" unless="no-javadoc">
+      <create-javadoc destdir="${sstableloader-javadoc.dir}">
+        <filesets>
+          <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+            <include name="org/apache/**/*.java"/>
+          </fileset>
+        </filesets>
+      </create-javadoc>
+    </target>
+
+    <target name="sstableloader-javadoc-jar" depends="sstableloader-javadoc" unless="no-javadoc" description="Assemble sstableloader JavaDoc JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-javadoc.jar" basedir="${sstableloader-javadoc.dir}"/>
+      <delete quiet="true" dir="${sstableloader-javadoc.dir}"/>
+    </target>
+
+    <target name="sstableloader-sources-jar" depends="init" description="Assemble sstableloader Sources JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-sources.jar">
+        <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+          <include name="org/apache/**/*.java"/>
+        </fileset>
+      </jar>
+    </target>
+

Review Comment:
   Just pushed something that I think works pretty well, hopefully that looks ok!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2876578460

   squashed into 1 commit, note that two tiny changes included in the commit that weren't there before:
   
   * Added a CHANGES.txt entry
   * Removed an extraneous space I caught in 'exit 2' at line 42 in bin/sstableloader.
   
   Other than this it is even with the original series of commits (87030caf) + commits since I merged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam closed pull request #3899: CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact
URL: https://github.com/apache/cassandra/pull/3899


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998553735


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   I don't get `this is likely a packaging issue as this script should not be installed.`
   
   This script should be still installed. It is just it will point to a new executable. It is just a delegate.
   
   Imho should be `Error: sstableloader has moved to the tools directory. Please install cassandra-tools package and try again.`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1955438117


##########
build.xml:
##########
@@ -2064,6 +2158,16 @@
     <install pomFile="${build.dir}/${final.name}.pom"
              file="${build.dir}/${final.name}-javadoc.jar"
              classifier="javadoc"/>
+
+    <!-- the cassandra-sstableloader jar -->
+    <install pomFile="${build.dir}/${final.name}-sstableloader.pom"

Review Comment:
   Tested this locally and it does appear to work:
   
   ```kotlin
   plugins {
       id("java")
   }
   
   group = "org.example"
   version = "1.0-SNAPSHOT"
   
   repositories {
       mavenCentral()
       mavenLocal()
   }
   
   dependencies {
       implementation("org.apache.cassandra:cassandra-sstableloader:5.1-SNAPSHOT")
       testImplementation(platform("org.junit:junit-bom:5.10.0"))
       testImplementation("org.junit.jupiter:junit-jupiter")
   }
   
   tasks.test {
       useJUnitPlatform()
   }
   ```
   
   ![Screenshot From 2025-02-13 19-37-11](https://github.com/user-attachments/assets/ad767c1b-4670-4d63-a215-9654fda4269a)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1955434962


##########
tools/sstableloader/test/unit/org/apache/cassandra/tools/LoaderOptionsTest.java:
##########
@@ -215,5 +215,40 @@ public void checkPermission(Permission perm, Object context)
             System.setSecurityManager(null);
         }
     }
+
+    // Copied from OfflineToolUtils

Review Comment:
   Ultimately ended up doing this instead of having sstableloader-test depend on test classes, as the test classes already depend on the sstableloader build, this didn't feel like it was worth doing, so just went for copying the minimal amount of shared code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2735127566

   rebased to get in sync with build.xml changes on trunk


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1955435174


##########
ide/nbproject/project.xml:
##########
@@ -35,6 +35,12 @@
                     <location>${project.dir}/tools/fqltool/src</location>
                     <encoding>UTF-8</encoding>
                 </source-folder>
+                <source-folder>

Review Comment:
   TODO: verify this actually works in netbeans.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998553735


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   I don't get `this is likely a packaging issue as this script should not be installed.`
   
   Imho should be `Note: sstableloader has moved to the tools directory. Please install cassandra-tools package and try again.`
   
   This script should be still installed. It is just it will point to a new executable. It is just a delegate.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998553735


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   I don't get `this is likely a packaging issue as this script should not be installed.`
   
   This script should be still installed. It is just it will point to a new executable. It is just a delegate.
   
   Imho should be `Note: sstableloader has moved to the tools directory. Please install cassandra-tools package and try again.`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998583185


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   hmmm thinking about it more, I see what we are doing here. This script is still part of tarball, it is just not "installed" as part of deb / rpm. When I look closely, I see this was installed to `/usr/bin/sstableloader` and `tools/bin/sstableloader` can not be installed there as well because it would replace what was already in /usr/bin/sstableloader. Huh.
   
   But still. I don't think this is _packaging issue_. Because e.g. a developer might still execute this script as he develops. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1965338822


##########
.build/parent-pom-template.xml:
##########
@@ -671,7 +671,7 @@
       <dependency>
         <groupId>org.apache.cassandra</groupId>
         <artifactId>cassandra-all</artifactId>
-        <version>4.1-alpha2-SNAPSHOT</version>

Review Comment:
   interesting 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1965730079


##########
.build/parent-pom-template.xml:
##########
@@ -671,7 +671,7 @@
       <dependency>
         <groupId>org.apache.cassandra</groupId>
         <artifactId>cassandra-all</artifactId>
-        <version>4.1-alpha2-SNAPSHOT</version>

Review Comment:
   yeah, this surprised me too!  Ultimately it wasn't being used because I don't think there was a dependency on it (until now).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2870271015

   Pleasantly surprising how trivial the merge from trunk after accord was, I put this off a little bit as thought i'd have conflicts everywhere but it was literally just a one-liner:
   
   ![image](https://github.com/user-attachments/assets/b758847a-6769-4152-bee6-c0d591c72915)
   
   There was also some changes to `LoaderOptionsTest` that I had to account for where some constants from `TlsTestUtils` were being used that we can't access anymore now that tests are in a separate module.  I figure it'd be fine to just not use the constants since don't think it is worth justifying adding the base test code to sstableloader's test classpath (3d431c0).
   
   Updated the Jenkins file in 87030ca and kicked off a ci test run to ensure everything is still good.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1955436329


##########
build.xml:
##########
@@ -2008,8 +2100,10 @@
   <classpathentry kind="src" output="${test.classes}" path="test/resources" />
   <classpathentry kind="src" path="tools/stress/src"/>
   <classpathentry kind="src" path="tools/fqltool/src"/>
+  <classpathentry kind="src" path="tools/sstableloader/src"/>

Review Comment:
   TODO: Verify this works in eclipse



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1997853263


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   Findings:
   
   debian:  `sstableloader` was already only bundled in cassandra-tools package, where `fqltool` and `cassandra-stress` were in the base cassandra_all
   
   redhat: `sstableloader` is bundled in the primary cassandra package.
   
   I think given the inconsistency, it's probably not best to fuss with this, and instead of focus on making it work as is.  Will push some changes shortly that accomplishes this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1997853263


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   I tried building both the debian and redhat package and noticed that `sstableloader`, `cassandra-stress` and `fqltool` were all present in the base package, not tools.  Given that is the case, I think its best to continue packaging things this way, and to update the builders to copy the sstableloader jar and the correct binary with it.  Will push a change that does this shortly.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1997815760


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   actually, its interesting;  It looks like both fqltool and cassandra-stress are packaged in the main cassandra package, not the tools package, despite the package designation.  This is a bit contrary to what I expected, but other tools like sstabledump, which don't have their own tools directory/library path are in the tools package.
   
   I think what I should do is just retain that behavior and keep sstableloader packaged in the cassandra package.  I'll just need to test it in a docker container to see that it does the right thing for both redhat and debian.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2000224360


##########
build.xml:
##########
@@ -2002,18 +2094,26 @@
   <classpathentry kind="src" output="${test.classes}" path="test/memory"/>
   <classpathentry kind="src" output="${test.classes}" path="test/burn"/>
   <classpathentry kind="src" output="${test.classes}" path="test/anttasks"/>
+  <classpathentry kind="src" output="${test.classes}" path="test/harry/main" />
   <classpathentry kind="src" output="${test.classes}" path="test/distributed"/>
   <classpathentry kind="src" output="${test.classes}" path="test/simulator/asm"/>
   <classpathentry kind="src" output="${test.classes}" path="test/simulator/main"/>
   <classpathentry kind="src" output="${test.classes}" path="test/resources" />
   <classpathentry kind="src" path="tools/stress/src"/>
   <classpathentry kind="src" path="tools/fqltool/src"/>
+  <classpathentry kind="src" path="tools/sstableloader/src"/>
   <classpathentry kind="src" output="build/test/stress-classes" path="tools/stress/test/unit" />
   <classpathentry kind="src" output="build/test/fqltool-classes" path="tools/fqltool/test/unit" />
-  <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
+  <classpathentry kind="src" output="build/test/sstableloader-classes" path="tools/sstableloader/test/unit" />
+  <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-11">
+      <attributes>
+        <attribute name="module" value="true"/>
+          <attribute name="add-exports" value="java.base/jdk.internal.misc=ALL-UNNAMED:java.base/jdk.internal.ref=ALL-UNNAMED:java.base/jdk.internal.util=ALL-UNNAMED:java.base/sun.nio.ch=ALL-UNNAMED:java.management.rmi/com.sun.jmx.remote.internal.rmi=ALL-UNNAMED:java.rmi/sun.rmi.registry=ALL-UNNAMED:java.rmi/sun.rmi.server=ALL-UNNAMED:java.rmi/sun.rmi.transport.tcp=ALL-UNNAMED:java.sql/java.sql=ALL-UNNAMED"/>

Review Comment:
   Just copied these from the definition elsewhere in build.xml.  I'm guessing we could programmatically generate this somehow, but wasn't sure if that was worth the trouble.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2875471362

   > could you please rebase once more? GitHub tells that it can't be rebased due to conflicts. Rebased on top of the current trunk (and optionally squashed) would be great.
   
   Yes, please rebase instead of merge.
   (We don't want the `Merge branch 'trunk' into CASSANDRA-20328` commits in this PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2766535000

   Moving this to ready for review now I think I've worked out that last issue with dtests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1997815760


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   actually, it's interesting;  It looks like both `fqltool` and `cassandra-stress` are packaged in the main cassandra package, not the tools package.  This is a bit contrary to what I expected, but other tools like sstabledump, which don't have their own tools directory/library path are in the tools package.
   
   I think what I should do is just retain that behavior and keep sstableloader packaged in the cassandra package.  I'll just need to test it in a docker container to see that it does the right thing for both redhat and debian.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2000223533


##########
build.xml:
##########
@@ -2025,6 +2125,9 @@
         </classpath>
     </taskdef>
     <mkdir dir=".settings" />
+    <echo file=".settings/org.eclipse.jdt.core.prefs"><![CDATA[eclipse.preferences.version=1

Review Comment:
   Needed because high-scale-lib provides a java.util package which causes eclipses to prevent compilation because the java.util module is detected in the JDK and on the classpath.
   
   ![Screenshot From 2025-03-18 00-16-46](https://github.com/user-attachments/assets/d0452186-764c-4407-869c-861e59a8c499)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998583185


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   hmmm thinking about it more, I see what we are doing here. This script is still part of tarball, it is just not "installed" as part of deb / rpm. When I look closely, I see this was installed to `/usr/bin/sstableloader` and `tools/bin/sstableloader` can not be installed there as well upon installing tools package because it would replace what was already in `/usr/bin/sstableloader`. Huh.
   
   But still. I don't think this is a _packaging issue_. Because e.g. a developer might still execute this script as he develops. I would just say that it moved to tools dir when not present. That's it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2077232664


##########
build.xml:
##########
@@ -664,6 +666,88 @@
         </testmacro>
     </target>
 
+    <!--
+        sstableloader build file
+        -->
+    <property name="sstableloader.build.src" value="${basedir}/tools/sstableloader/src" />
+    <property name="sstableloader.test.src" value="${basedir}/tools/sstableloader/test/unit" />
+    <property name="sstableloader.build.classes" value="${build.classes}/sstableloader" />
+    <property name="sstableloader.test.classes" value="${build.dir}/test/sstableloader-classes" />
+    <property name="sstableloader.manifest" value="${sstableloader.build.classes}/MANIFEST.MF" />
+
+    <target name="sstableloader-build-test" depends="sstableloader-build" description="Compile sstableloader tests">
+        <javac debug="true" debuglevel="${debuglevel}" destdir="${sstableloader.test.classes}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               includeantruntime="false" encoding="utf-8">
+            <classpath>
+                <path refid="cassandra.classpath.test"/>
+                <pathelement location="${sstableloader.build.classes}" />
+            </classpath>
+            <src path="${sstableloader.test.src}"/>
+        </javac>
+    </target>
+
+    <target name="sstableloader-build" depends="build" description="build sstableloader">
+        <antcall target="_sstableloader_build"/>
+    </target>
+
+    <target name="_sstableloader_build">
+        <mkdir dir="${sstableloader.build.classes}" />
+        <javac compiler="modern" debug="true" debuglevel="${debuglevel}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               encoding="utf-8" destdir="${sstableloader.build.classes}" includeantruntime="true">
+            <src path="${sstableloader.build.src}" />
+            <classpath>
+                <path refid="cassandra.classpath" />
+            </classpath>
+        </javac>
+    </target>
+
+    <target name="sstableloader-test" depends="sstableloader-build-test, maybe-build-test" description="Runs sstableloader tests">
+        <testmacro inputdir="${sstableloader.test.src}"
+                       timeout="${test.timeout}">
+        </testmacro>
+    </target>
+
+    <target name="sstableloader-jar"
+            depends="sstableloader-build"
+            description="Assemble Cassandra sstableloader JAR files">
+
+        <!-- sstableloader jar -->
+        <manifest file="${sstableloader.manifest}">
+            <attribute name="Built-By" value="Apache Cassandra Project"/>
+            <attribute name="Main-Class" value="org.apache.cassandra.tools.BulkLoader"/>
+        </manifest>
+        <mkdir dir="${sstableloader.build.classes}/META-INF" />
+        <mkdir dir="${build.dir}/tools/lib/" />
+        <jar destfile="${build.dir}/tools/lib/sstableloader.jar" manifest="${sstableloader.manifest}">
+            <fileset dir="${sstableloader.build.classes}"/>
+        </jar>
+    </target>
+
+    <target name="sstableloader-javadoc" depends="build" description="Create javadoc" unless="no-javadoc">
+      <create-javadoc destdir="${sstableloader-javadoc.dir}">
+        <filesets>
+          <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+            <include name="org/apache/**/*.java"/>
+          </fileset>
+        </filesets>
+      </create-javadoc>
+    </target>
+
+    <target name="sstableloader-javadoc-jar" depends="sstableloader-javadoc" unless="no-javadoc" description="Assemble sstableloader JavaDoc JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-javadoc.jar" basedir="${sstableloader-javadoc.dir}"/>
+      <delete quiet="true" dir="${sstableloader-javadoc.dir}"/>
+    </target>
+
+    <target name="sstableloader-sources-jar" depends="init" description="Assemble sstableloader Sources JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-sources.jar">
+        <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+          <include name="org/apache/**/*.java"/>
+        </fileset>
+      </jar>
+    </target>
+

Review Comment:
   looks good.



##########
.build/sstableloader-deps-template.xml:
##########


Review Comment:
   looks good.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2873062526

   @tolbertam could you please rebase once more? GitHub tells that it can't be rebased due to conflicts. Rebased on top of the current trunk (and optionally squashed) would be great.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2030989522


##########
.build/sstableloader-deps-template.xml:
##########


Review Comment:
   i've never settled happy on the naming structure of these new files.
   
   i think i'd prefer `.build/sstableloader-deps-maven-pom.xml` 
   as something that was more intuitive.  "template" doesn't tell us much (it's not that important to know upfront that this file gets filtered during the build), but immediate feedback that this is our maven pom dependency declarations is valuable.
   
   but any rename should apply consistently to all such files.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2814223683

   this is going to need a rebasing with the accord merge which i'll look into, but there are some things that accord changed about the build that I really like and I think could motivate further change.  Will spend some time wrapping my head around this and come back to this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1965353984


##########
tools/bin/sstableloader:
##########
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+if [ "x$CASSANDRA_INCLUDE" = "x" ]; then

Review Comment:
   tools are in a different package a user has to install so when we move sstableloader among tools, then, suddenly, the script in `bin/sstableloader` will point to a non-existing `tools/bin/sstableloader` (unless a user installs it).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1999180681


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   ok just keep it like it is 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2000226951


##########
build.xml:
##########
@@ -1986,7 +2078,7 @@
     </path>
     <pathconvert property="eclipse-libs-list" refid="eclipse-project-libs-path" pathsep="${line.separator}">
         <mapper>
-            <regexpmapper from="^(.*)$$" to='&lt;classpathentry kind="lib" path="\1\" \/&gt;'/>
+            <regexpmapper from="^(.*)$$" to='  &lt;classpathentry kind="lib" path="\1" /&gt;'/>

Review Comment:
   This was previously generating invalid xml in the .classpath file:
   
   ```
   <classpathentry kind="lib" path="/home/tolbertam/Documents/Projects/cassandra/lib/ST4-4.0.8.jar\" \/>
   ```
   
   In particular the `\/>` part is invalid but I expect `...jar\"` probably would throw things off too.  Also added two whitespaces to the beginning to make the spacing look right (not required, just prettier). 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998971817


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   That's right,  I think given stress and fqltool are bundled in the main package already (even though they have separate builds into tools), we might as well do the same for sstableloader.



##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   That's right,  I think given stress and fqltool are bundled in the main distribution package already (even though they have separate builds into tools), we might as well do the same for sstableloader.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998601162


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   @tolbertam I see. So the plan is to keep packaging sstableloader as is, it would be just a different class path etc. Right?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2032270467


##########
.build/sstableloader-deps-template.xml:
##########


Review Comment:
   I agree that it definitely is more apparent what the file is for, will make that change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2030979391


##########
build.xml:
##########
@@ -664,6 +666,88 @@
         </testmacro>
     </target>
 
+    <!--
+        sstableloader build file
+        -->
+    <property name="sstableloader.build.src" value="${basedir}/tools/sstableloader/src" />
+    <property name="sstableloader.test.src" value="${basedir}/tools/sstableloader/test/unit" />
+    <property name="sstableloader.build.classes" value="${build.classes}/sstableloader" />
+    <property name="sstableloader.test.classes" value="${build.dir}/test/sstableloader-classes" />
+    <property name="sstableloader.manifest" value="${sstableloader.build.classes}/MANIFEST.MF" />
+
+    <target name="sstableloader-build-test" depends="sstableloader-build" description="Compile sstableloader tests">
+        <javac debug="true" debuglevel="${debuglevel}" destdir="${sstableloader.test.classes}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               includeantruntime="false" encoding="utf-8">
+            <classpath>
+                <path refid="cassandra.classpath.test"/>
+                <pathelement location="${sstableloader.build.classes}" />
+            </classpath>
+            <src path="${sstableloader.test.src}"/>
+        </javac>
+    </target>
+
+    <target name="sstableloader-build" depends="build" description="build sstableloader">
+        <antcall target="_sstableloader_build"/>
+    </target>
+
+    <target name="_sstableloader_build">
+        <mkdir dir="${sstableloader.build.classes}" />
+        <javac compiler="modern" debug="true" debuglevel="${debuglevel}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               encoding="utf-8" destdir="${sstableloader.build.classes}" includeantruntime="true">
+            <src path="${sstableloader.build.src}" />
+            <classpath>
+                <path refid="cassandra.classpath" />
+            </classpath>
+        </javac>
+    </target>
+
+    <target name="sstableloader-test" depends="sstableloader-build-test, maybe-build-test" description="Runs sstableloader tests">
+        <testmacro inputdir="${sstableloader.test.src}"
+                       timeout="${test.timeout}">
+        </testmacro>
+    </target>
+
+    <target name="sstableloader-jar"
+            depends="sstableloader-build"
+            description="Assemble Cassandra sstableloader JAR files">
+
+        <!-- sstableloader jar -->
+        <manifest file="${sstableloader.manifest}">
+            <attribute name="Built-By" value="Apache Cassandra Project"/>
+            <attribute name="Main-Class" value="org.apache.cassandra.tools.BulkLoader"/>
+        </manifest>
+        <mkdir dir="${sstableloader.build.classes}/META-INF" />
+        <mkdir dir="${build.dir}/tools/lib/" />
+        <jar destfile="${build.dir}/tools/lib/sstableloader.jar" manifest="${sstableloader.manifest}">
+            <fileset dir="${sstableloader.build.classes}"/>
+        </jar>
+    </target>
+
+    <target name="sstableloader-javadoc" depends="build" description="Create javadoc" unless="no-javadoc">
+      <create-javadoc destdir="${sstableloader-javadoc.dir}">
+        <filesets>
+          <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+            <include name="org/apache/**/*.java"/>
+          </fileset>
+        </filesets>
+      </create-javadoc>
+    </target>
+
+    <target name="sstableloader-javadoc-jar" depends="sstableloader-javadoc" unless="no-javadoc" description="Assemble sstableloader JavaDoc JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-javadoc.jar" basedir="${sstableloader-javadoc.dir}"/>
+      <delete quiet="true" dir="${sstableloader-javadoc.dir}"/>
+    </target>
+
+    <target name="sstableloader-sources-jar" depends="init" description="Assemble sstableloader Sources JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-sources.jar">
+        <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+          <include name="org/apache/**/*.java"/>
+        </fileset>
+      </jar>
+    </target>
+

Review Comment:
   what about putting this to a `tools/sstableloader/build.xml` if we permit cyclic imports or otherwise to a `.build/build-sstableloader.xml`  ,and importing it here ?
   
   This is generally along the lines of what's being done breaking out encapsulated groupings of targets in the .build/*.xml sub-files.  Given this a submodule,  if cyclic imports work (each file imports each other, because these new targets reference `init` and `build`) then it might make sense to put it in that submodule folder rather than under .build/



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1997808715


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   I suppose now that we're bundling sstableloader with tools, using it will now require the cassandra-tools package, where previously it was bundled with cassandra.  I hope this is the kind of change we can make between 5.0 and whatever the next release?
   
   We could in theory still bundle sstableloader with the core package, it's just that it will ultimately its library will live in the tools directory.
   
   I'll go with bundling it in the tools package and updating this scrip to check for presence of the script and erroring out with a message as you suggest.  I will also add something to news for this.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1997853263


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   I tried building both the debian and redhat package and noticed that `sstableloader`, `cassandra-stress` and `fqltool` were all present in the base package, not tools.  Given that is the case, I think its best to continue packaging things this way, and to update the builders to copy the sstableloader jar and the correct binary with it.  Will push a change that does this shortly.
   
   Another thing I noticed was that these jars are always present on the classpath, even for cassandra server.  When I implement [CASSANDRA-20329](https://issues.apache.org/jira/browse/CASSANDRA-20329) I'll make sure that is no longer the case.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2032269935


##########
build.xml:
##########
@@ -664,6 +666,88 @@
         </testmacro>
     </target>
 
+    <!--
+        sstableloader build file
+        -->
+    <property name="sstableloader.build.src" value="${basedir}/tools/sstableloader/src" />
+    <property name="sstableloader.test.src" value="${basedir}/tools/sstableloader/test/unit" />
+    <property name="sstableloader.build.classes" value="${build.classes}/sstableloader" />
+    <property name="sstableloader.test.classes" value="${build.dir}/test/sstableloader-classes" />
+    <property name="sstableloader.manifest" value="${sstableloader.build.classes}/MANIFEST.MF" />
+
+    <target name="sstableloader-build-test" depends="sstableloader-build" description="Compile sstableloader tests">
+        <javac debug="true" debuglevel="${debuglevel}" destdir="${sstableloader.test.classes}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               includeantruntime="false" encoding="utf-8">
+            <classpath>
+                <path refid="cassandra.classpath.test"/>
+                <pathelement location="${sstableloader.build.classes}" />
+            </classpath>
+            <src path="${sstableloader.test.src}"/>
+        </javac>
+    </target>
+
+    <target name="sstableloader-build" depends="build" description="build sstableloader">
+        <antcall target="_sstableloader_build"/>
+    </target>
+
+    <target name="_sstableloader_build">
+        <mkdir dir="${sstableloader.build.classes}" />
+        <javac compiler="modern" debug="true" debuglevel="${debuglevel}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               encoding="utf-8" destdir="${sstableloader.build.classes}" includeantruntime="true">
+            <src path="${sstableloader.build.src}" />
+            <classpath>
+                <path refid="cassandra.classpath" />
+            </classpath>
+        </javac>
+    </target>
+
+    <target name="sstableloader-test" depends="sstableloader-build-test, maybe-build-test" description="Runs sstableloader tests">
+        <testmacro inputdir="${sstableloader.test.src}"
+                       timeout="${test.timeout}">
+        </testmacro>
+    </target>
+
+    <target name="sstableloader-jar"
+            depends="sstableloader-build"
+            description="Assemble Cassandra sstableloader JAR files">
+
+        <!-- sstableloader jar -->
+        <manifest file="${sstableloader.manifest}">
+            <attribute name="Built-By" value="Apache Cassandra Project"/>
+            <attribute name="Main-Class" value="org.apache.cassandra.tools.BulkLoader"/>
+        </manifest>
+        <mkdir dir="${sstableloader.build.classes}/META-INF" />
+        <mkdir dir="${build.dir}/tools/lib/" />
+        <jar destfile="${build.dir}/tools/lib/sstableloader.jar" manifest="${sstableloader.manifest}">
+            <fileset dir="${sstableloader.build.classes}"/>
+        </jar>
+    </target>
+
+    <target name="sstableloader-javadoc" depends="build" description="Create javadoc" unless="no-javadoc">
+      <create-javadoc destdir="${sstableloader-javadoc.dir}">
+        <filesets>
+          <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+            <include name="org/apache/**/*.java"/>
+          </fileset>
+        </filesets>
+      </create-javadoc>
+    </target>
+
+    <target name="sstableloader-javadoc-jar" depends="sstableloader-javadoc" unless="no-javadoc" description="Assemble sstableloader JavaDoc JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-javadoc.jar" basedir="${sstableloader-javadoc.dir}"/>
+      <delete quiet="true" dir="${sstableloader-javadoc.dir}"/>
+    </target>
+
+    <target name="sstableloader-sources-jar" depends="init" description="Assemble sstableloader Sources JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-sources.jar">
+        <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+          <include name="org/apache/**/*.java"/>
+        </fileset>
+      </jar>
+    </target>
+

Review Comment:
   that sounds good to me, i'll give this a try and see how it goes :+1:



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2030979391


##########
build.xml:
##########
@@ -664,6 +666,88 @@
         </testmacro>
     </target>
 
+    <!--
+        sstableloader build file
+        -->
+    <property name="sstableloader.build.src" value="${basedir}/tools/sstableloader/src" />
+    <property name="sstableloader.test.src" value="${basedir}/tools/sstableloader/test/unit" />
+    <property name="sstableloader.build.classes" value="${build.classes}/sstableloader" />
+    <property name="sstableloader.test.classes" value="${build.dir}/test/sstableloader-classes" />
+    <property name="sstableloader.manifest" value="${sstableloader.build.classes}/MANIFEST.MF" />
+
+    <target name="sstableloader-build-test" depends="sstableloader-build" description="Compile sstableloader tests">
+        <javac debug="true" debuglevel="${debuglevel}" destdir="${sstableloader.test.classes}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               includeantruntime="false" encoding="utf-8">
+            <classpath>
+                <path refid="cassandra.classpath.test"/>
+                <pathelement location="${sstableloader.build.classes}" />
+            </classpath>
+            <src path="${sstableloader.test.src}"/>
+        </javac>
+    </target>
+
+    <target name="sstableloader-build" depends="build" description="build sstableloader">
+        <antcall target="_sstableloader_build"/>
+    </target>
+
+    <target name="_sstableloader_build">
+        <mkdir dir="${sstableloader.build.classes}" />
+        <javac compiler="modern" debug="true" debuglevel="${debuglevel}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               encoding="utf-8" destdir="${sstableloader.build.classes}" includeantruntime="true">
+            <src path="${sstableloader.build.src}" />
+            <classpath>
+                <path refid="cassandra.classpath" />
+            </classpath>
+        </javac>
+    </target>
+
+    <target name="sstableloader-test" depends="sstableloader-build-test, maybe-build-test" description="Runs sstableloader tests">
+        <testmacro inputdir="${sstableloader.test.src}"
+                       timeout="${test.timeout}">
+        </testmacro>
+    </target>
+
+    <target name="sstableloader-jar"
+            depends="sstableloader-build"
+            description="Assemble Cassandra sstableloader JAR files">
+
+        <!-- sstableloader jar -->
+        <manifest file="${sstableloader.manifest}">
+            <attribute name="Built-By" value="Apache Cassandra Project"/>
+            <attribute name="Main-Class" value="org.apache.cassandra.tools.BulkLoader"/>
+        </manifest>
+        <mkdir dir="${sstableloader.build.classes}/META-INF" />
+        <mkdir dir="${build.dir}/tools/lib/" />
+        <jar destfile="${build.dir}/tools/lib/sstableloader.jar" manifest="${sstableloader.manifest}">
+            <fileset dir="${sstableloader.build.classes}"/>
+        </jar>
+    </target>
+
+    <target name="sstableloader-javadoc" depends="build" description="Create javadoc" unless="no-javadoc">
+      <create-javadoc destdir="${sstableloader-javadoc.dir}">
+        <filesets>
+          <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+            <include name="org/apache/**/*.java"/>
+          </fileset>
+        </filesets>
+      </create-javadoc>
+    </target>
+
+    <target name="sstableloader-javadoc-jar" depends="sstableloader-javadoc" unless="no-javadoc" description="Assemble sstableloader JavaDoc JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-javadoc.jar" basedir="${sstableloader-javadoc.dir}"/>
+      <delete quiet="true" dir="${sstableloader-javadoc.dir}"/>
+    </target>
+
+    <target name="sstableloader-sources-jar" depends="init" description="Assemble sstableloader Sources JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-sources.jar">
+        <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+          <include name="org/apache/**/*.java"/>
+        </fileset>
+      </jar>
+    </target>
+

Review Comment:
   what about putting this to a `tools/sstableloader/build.xml` if we permit cyclic imports or otherwise to a `.build/build-sstableloader.xml`  ,and importing it here ?
   
   This is generally along the lines of what's being done breaking out encapsulated groupings of targets in the .build/*.xml sub-files.  Given this a submodule,  if cyclic imports work (each file imports each other, because these new targets reference `init` and `build`) then it makes sense to put it in that submodule folder rather than under .build/



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998583185


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   hmmm thinking about it more, I see what we are doing here. This script is still part of tarball, it is just not "installed" as part of deb / rpm. When I look closely, I see this was installed to `/usr/bin/sstableloader` and `tools/bin/sstableloader` can not be installed there as well upon installing tools package because it would replace what was already in `/usr/bin/sstableloader`. Huh.
   
   But still. I don't think this is a _packaging issue_. Because e.g. a developer might still execute this script as he develops. I would just say that it moved too tools dir when not present. That's it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2000233372


##########
build.xml:
##########
@@ -2008,8 +2100,10 @@
   <classpathentry kind="src" output="${test.classes}" path="test/resources" />
   <classpathentry kind="src" path="tools/stress/src"/>
   <classpathentry kind="src" path="tools/fqltool/src"/>
+  <classpathentry kind="src" path="tools/sstableloader/src"/>

Review Comment:
   This seems to work as I can run `SSTableLoaderTest` and all the tests pass, but I had to make eclipse work again as it looks like it's currently broken for a number of reasons (see other comments).
   
   If you think these changes are out of scope let me know and I can do them as a separate jira/PR.
   
   Note that after doing this there were still a few things i needed to do:
   
   1. Add `-DallowAttachSelf=true` to the VM run arguments for any run configuration.
   2. Update `SortedTableWriter.AbstractIndexWriter` to be public as otherwise I get this error:
   
   ```
   Caused by: java.lang.Error: Unresolved compilation problem: 
   	The type SortedTableWriter.AbstractIndexWriter is not visible
   
   	at org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.<init>(BigTableWriter.java:237)
   	at org.apache.cassandra.io.sstable.format.big.BigTableWriter$Builder.openIndexWriter(BigTableWriter.java:405)
   	at org.apache.cassandra.io.sstable.format.big.BigTableWriter$Builder.openIndexWriter(BigTableWriter.java:1)
   ```
   
   Change made:
   ```diff
   diff --git a/src/java/org/apache/cassandra/io/sstable/format/SortedTableWriter.java b/src/java/org/apache/cassandra/io/sstable/format/SortedTableWriter.java
   index 4fca9ca181..67ae1c7402 100644
   --- a/src/java/org/apache/cassandra/io/sstable/format/SortedTableWriter.java
   +++ b/src/java/org/apache/cassandra/io/sstable/format/SortedTableWriter.java
   @@ -427,7 +427,7 @@ public abstract class SortedTableWriter<P extends SortedTablePartitionWriter, I
            }
        }
   
   -    protected static abstract class AbstractIndexWriter extends AbstractTransactional implements Transactional
   +    public static abstract class AbstractIndexWriter extends AbstractTransactional implements Transactional
        {
            protected final Descriptor descriptor;
            protected final TableMetadataRef metadata;
   ```
   
   I suspect this isn't actually necessary, I think maybe what is happening is the code ends up somewhere else on the build path, as I see duplication in the jars being added, so the same may be happening on the classpath.  I'm not sure how much further I should to take this other than to ensure I can at a base level get things working.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998606886


##########
tools/sstableloader/src/org/apache/cassandra/tools/LoaderOptions.java:
##########
@@ -79,7 +78,6 @@ public class LoaderOptions
      * provide options instead of using these constant fields.
      * @deprecated Use {@code throttle-mib} instead
      */
-    /** @deprecated See CASSANDRA-17677 */

Review Comment:
   @tolbertam are you sure about this? The fact we moved the class from dir to dir does not mean this stopped to be relevant. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998973038


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   Yeah, thats a good idea, keeps things simple to make it a dumb proxy, will change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1965350133


##########
tools/bin/sstableloader:
##########
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+if [ "x$CASSANDRA_INCLUDE" = "x" ]; then

Review Comment:
   @tolbertam please check `redhat/cassandra.spec`. I know that `bin/sstableloader` points to this one, I just wonder what changes, if anything, if we introduce this script among tools. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1965353984


##########
tools/bin/sstableloader:
##########
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+if [ "x$CASSANDRA_INCLUDE" = "x" ]; then

Review Comment:
   tools are in a different package a user has to install so when we move sstableloader among tools, then, suddenly, the script in `bin/sstableloader` will point to a non-existing `tools/bin/sstableloader`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041157473


##########
build.xml:
##########
@@ -360,26 +360,38 @@
         </fileset>
     </path>
 
-  <macrodef name="create-javadoc">
-    <attribute name="destdir"/>
-    <element name="filesets"/>
-    <sequential>
-      <javadoc destdir="@{destdir}" author="true" version="true" use="true"
-        windowtitle="${ant.project.name} API" classpathref="cassandra.classpath"
-        bottom="Copyright &amp;copy; 2009- The Apache Software Foundation"
-        useexternalfile="yes" encoding="UTF-8" failonerror="false"
-        maxmemory="256m" additionalparam="${jdk11plus-javac-exports}">
-        <filesets/>
-      </javadoc>
-      <fail message="javadoc failed">
-        <condition>
-            <not>
-                <available file="@{destdir}/index-all.html" />
-            </not>
-        </condition>
-      </fail>
-    </sequential>
-  </macrodef>
+    <macrodef name="create-javadoc">
+      <attribute name="destdir"/>
+      <element name="filesets"/>
+      <sequential>
+        <javadoc destdir="@{destdir}" author="true" version="true" use="true"
+          windowtitle="${ant.project.name} API" classpathref="cassandra.classpath"
+          bottom="Copyright &amp;copy; 2009- The Apache Software Foundation"
+          useexternalfile="yes" encoding="UTF-8" failonerror="false"
+          maxmemory="256m" additionalparam="${jdk11plus-javac-exports}">
+          <filesets/>
+        </javadoc>
+        <fail message="javadoc failed">
+          <condition>
+              <not>
+                  <available file="@{destdir}/index-all.html" />
+              </not>
+          </condition>
+        </fail>
+      </sequential>
+    </macrodef>

Review Comment:
   no changes here other than correcting indentation (can remove if needed)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041159180


##########
build.xml:
##########
@@ -1211,10 +1215,12 @@
           <pathelement path="${java.class.path}"/>
           <pathelement location="${stress.build.classes}"/>
           <pathelement location="${fqltool.build.classes}"/>
+          <pathelement location="${sstableloader.build.classes}"/>

Review Comment:
   This is a bit suboptimal, but because `sstableloader-test` makes use of this macro we need to include the classes here as well.  While not perfect it doesn't feel too problematic to have to define these here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2852875323

   i'll work on rebasing this this week.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041142169


##########
build.xml:
##########
@@ -664,6 +666,88 @@
         </testmacro>
     </target>
 
+    <!--
+        sstableloader build file
+        -->
+    <property name="sstableloader.build.src" value="${basedir}/tools/sstableloader/src" />
+    <property name="sstableloader.test.src" value="${basedir}/tools/sstableloader/test/unit" />
+    <property name="sstableloader.build.classes" value="${build.classes}/sstableloader" />
+    <property name="sstableloader.test.classes" value="${build.dir}/test/sstableloader-classes" />
+    <property name="sstableloader.manifest" value="${sstableloader.build.classes}/MANIFEST.MF" />
+
+    <target name="sstableloader-build-test" depends="sstableloader-build" description="Compile sstableloader tests">
+        <javac debug="true" debuglevel="${debuglevel}" destdir="${sstableloader.test.classes}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               includeantruntime="false" encoding="utf-8">
+            <classpath>
+                <path refid="cassandra.classpath.test"/>
+                <pathelement location="${sstableloader.build.classes}" />
+            </classpath>
+            <src path="${sstableloader.test.src}"/>
+        </javac>
+    </target>
+
+    <target name="sstableloader-build" depends="build" description="build sstableloader">
+        <antcall target="_sstableloader_build"/>
+    </target>
+
+    <target name="_sstableloader_build">
+        <mkdir dir="${sstableloader.build.classes}" />
+        <javac compiler="modern" debug="true" debuglevel="${debuglevel}"
+               source="${ant.java.version}" target="${ant.java.version}"
+               encoding="utf-8" destdir="${sstableloader.build.classes}" includeantruntime="true">
+            <src path="${sstableloader.build.src}" />
+            <classpath>
+                <path refid="cassandra.classpath" />
+            </classpath>
+        </javac>
+    </target>
+
+    <target name="sstableloader-test" depends="sstableloader-build-test, maybe-build-test" description="Runs sstableloader tests">
+        <testmacro inputdir="${sstableloader.test.src}"
+                       timeout="${test.timeout}">
+        </testmacro>
+    </target>
+
+    <target name="sstableloader-jar"
+            depends="sstableloader-build"
+            description="Assemble Cassandra sstableloader JAR files">
+
+        <!-- sstableloader jar -->
+        <manifest file="${sstableloader.manifest}">
+            <attribute name="Built-By" value="Apache Cassandra Project"/>
+            <attribute name="Main-Class" value="org.apache.cassandra.tools.BulkLoader"/>
+        </manifest>
+        <mkdir dir="${sstableloader.build.classes}/META-INF" />
+        <mkdir dir="${build.dir}/tools/lib/" />
+        <jar destfile="${build.dir}/tools/lib/sstableloader.jar" manifest="${sstableloader.manifest}">
+            <fileset dir="${sstableloader.build.classes}"/>
+        </jar>
+    </target>
+
+    <target name="sstableloader-javadoc" depends="build" description="Create javadoc" unless="no-javadoc">
+      <create-javadoc destdir="${sstableloader-javadoc.dir}">
+        <filesets>
+          <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+            <include name="org/apache/**/*.java"/>
+          </fileset>
+        </filesets>
+      </create-javadoc>
+    </target>
+
+    <target name="sstableloader-javadoc-jar" depends="sstableloader-javadoc" unless="no-javadoc" description="Assemble sstableloader JavaDoc JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-javadoc.jar" basedir="${sstableloader-javadoc.dir}"/>
+      <delete quiet="true" dir="${sstableloader-javadoc.dir}"/>
+    </target>
+
+    <target name="sstableloader-sources-jar" depends="init" description="Assemble sstableloader Sources JAR file">
+      <jar jarfile="${build.dir}/${final.name}-sstableloader-sources.jar">
+        <fileset dir="${sstableloader.build.src}" defaultexcludes="yes">
+          <include name="org/apache/**/*.java"/>
+        </fileset>
+      </jar>
+    </target>
+

Review Comment:
   I was skeptical but it looks like circular imports do work!   I'm going to do a little more refactoring to move out sstableloader-y stuff from common tasks and into their own tasks but should have something committed in a little bit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041157922


##########
build.xml:
##########
@@ -2008,8 +2100,10 @@
   <classpathentry kind="src" output="${test.classes}" path="test/resources" />
   <classpathentry kind="src" path="tools/stress/src"/>
   <classpathentry kind="src" path="tools/fqltool/src"/>
+  <classpathentry kind="src" path="tools/sstableloader/src"/>

Review Comment:
   Can ignore this commentary (doing the unrelated eclipse changes in #4020 instead)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1997815760


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   actually, it's interesting;  It looks like both `fqltool` and `cassandra-stress` are packaged in the main cassandra package, not the tools package.  This is a bit contrary to what I expected, but other tools like sstabledump, which don't have their own tools directory/library path are in the tools package.
   
   I think what I should do is just retain that behavior and keep sstableloader packaged in the cassandra package.  I'll just need to test it in a docker container to see that it does the right thing for both redhat and debian.
   
   We'll also need to make sure that the right `sstableloader` binary is copied to `/usr/bin`, which should hopefully be a trivial change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1997815760


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   actually, its interesting;  It looks like both `fqltool` and `cassandra-stress` are packaged in the main cassandra package, not the tools package.  This is a bit contrary to what I expected, but other tools like sstabledump, which don't have their own tools directory/library path are in the tools package.
   
   I think what I should do is just retain that behavior and keep sstableloader packaged in the cassandra package.  I'll just need to test it in a docker container to see that it does the right thing for both redhat and debian.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998583185


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   hmmm thinking about it more, I see what we are doing here. This script is still part of tarball, it is just not "installed" as part of deb / rpm. When I look closely, I see this was installed to '/usr/bin/sstableloader` and `tools/bin/sstableloader` can not be installed there as well because it would replace what was already in /usr/bin/sstableloader. Huh.
   
   But still. I don't think this is _packaging issue_. Because e.g. a developer might still execute this script as he develops. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998551372


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \
+issue as this script should not be installed." >&2
+    exit 1
+else
+    echo "Warning: sstableloader has been moved to the tools directory. \

Review Comment:
   @tolbertam I am not sure we want to do this. The logic should be "if the script in tools does not exist, tell user that it moved to tools and they should install that package, otherwise run it".
   
   In other words, there is no reason to emit a warning telling a user that script moved and then execute that script. Why should a user care it moved if it works? I would just remove that "else" branch completely.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998547027


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   @tolbertam why have you removed all of this?
   
   ```
   if [ -z "$CLASSPATH" ]; then
       echo "You must set the CLASSPATH var" >&2
       exit 1
   fi
   
   if [ "x$MAX_HEAP_SIZE" = "x" ]; then
       MAX_HEAP_SIZE="256M"
   fi
   ```
   edit:
   
   @tolbertam 
   
   Aha I see, because it is all moved under tools/bin/sstableloader. Interesting. That way we might get rid of _everything_ above, no? Just check the script's existence and then call it with all the args.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1965356704


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   @tolbertam  I would check the existence of this script because sstableloader is a tool which is / should be installed by a different package (redhat/cassandra.spec).
   
   If it is not present, then we need to tell a user that they should install tools package.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1965730904


##########
tools/bin/sstableloader:
##########
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+if [ "x$CASSANDRA_INCLUDE" = "x" ]; then

Review Comment:
   ah, this is not something I had thought about yet, I'll see what I can do about this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2889117877

   Committed as f80a234d1ec33dbe927bc8af28f1a290335819a3, thanks all!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1955532046


##########
build.xml:
##########
@@ -64,6 +64,7 @@
     <property name="build.classes" value="${build.dir}/classes"/>
     <property name="build.classes.main" value="${build.classes}/main" />
     <property name="javadoc.dir" value="${build.dir}/javadoc"/>
+    <property name="sstableloader-javadoc.dir" value="${build.dir}/sstableloader-javadoc"/>

Review Comment:
   Move to ~660 where other sstableloader props are, rename to `sstableloader.javadoc.dir` for consistency.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1997815760


##########
bin/sstableloader:
##########
@@ -32,18 +35,6 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
-
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
-fi
-
-"$JAVA" $JAVA_AGENT -ea -cp "$CLASSPATH" $JVM_OPTS -Xmx$MAX_HEAP_SIZE \
-        -Dcassandra.storagedir="$cassandra_storagedir" \
-        -Dlogback.configurationFile=logback-tools.xml \
-        org.apache.cassandra.tools.BulkLoader "$@"
+"$CASSANDRA_HOME/tools/bin/sstableloader" "$@"

Review Comment:
   actually, its interesting;  It looks like both fqltool and cassandra-stress are packaged in the main cassandra package, not the tools package, despite the package designation.  This is a bit contrary to what I expected, but other tools like sstabledump, which don't have their own tools directory/library path are in the tools package.
   
   I think what I should do is just retain that behavior and keep sstableloader packaged in the cassandra package.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998583185


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   hmmm thinking about it more, I see what we are doing here. This script is still part of tarball, it is just not "installed" as part of deb / rpm. When I look closely, I see this was installed to `/usr/bin/sstableloader` and `tools/bin/sstableloader` can not be installed there as well upon installing tools package because it would replace what was already in `/usr/bin/sstableloader`. Huh.
   
   But still. I don't think this is _packaging issue_. Because e.g. a developer might still execute this script as he develops. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998547027


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   @tolbertam why have you removed all of this?
   
   ```
   if [ -z "$CLASSPATH" ]; then
       echo "You must set the CLASSPATH var" >&2
       exit 1
   fi
   
   if [ "x$MAX_HEAP_SIZE" = "x" ]; then
       MAX_HEAP_SIZE="256M"
   fi
   ```
   edit:
   
   @tolbertam 
   
   Aha I see, because it is all moved under tools/bin/sstableloader. Interesting. That way we might get rid of _everything_ above, no? Just check the script's existence and then call it with all the args. This script should be just a "dumb proxy" (plus checking if the target script exists).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998547027


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   @tolbertam why have you removed all of this?
   
   ```
   if [ -z "$CLASSPATH" ]; then
       echo "You must set the CLASSPATH var" >&2
       exit 1
   fi
   
   if [ "x$MAX_HEAP_SIZE" = "x" ]; then
       MAX_HEAP_SIZE="256M"
   fi
   ```
   edit:
   
   @tolbertam 
   
   Aha I see, because it is all moved under tools/bin/sstableloader. Interesting. That way we might get rid of _everything_ above, no? Just check the script's existence and then call it with all the args. This script should be just a "dumb proxy" (plus checking if the target script exists".



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1965730079


##########
.build/parent-pom-template.xml:
##########
@@ -671,7 +671,7 @@
       <dependency>
         <groupId>org.apache.cassandra</groupId>
         <artifactId>cassandra-all</artifactId>
-        <version>4.1-alpha2-SNAPSHOT</version>

Review Comment:
   yeah, this surprised me too!  Ultimately it wasn't being used because I don't think it was used until now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2032270467


##########
.build/sstableloader-deps-template.xml:
##########


Review Comment:
   that sounds good to me, I agree that it definitely is more apparent what the file is for.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041157564


##########
build.xml:
##########
@@ -360,26 +360,38 @@
         </fileset>
     </path>
 
-  <macrodef name="create-javadoc">
-    <attribute name="destdir"/>
-    <element name="filesets"/>
-    <sequential>
-      <javadoc destdir="@{destdir}" author="true" version="true" use="true"
-        windowtitle="${ant.project.name} API" classpathref="cassandra.classpath"
-        bottom="Copyright &amp;copy; 2009- The Apache Software Foundation"
-        useexternalfile="yes" encoding="UTF-8" failonerror="false"
-        maxmemory="256m" additionalparam="${jdk11plus-javac-exports}">
-        <filesets/>
-      </javadoc>
-      <fail message="javadoc failed">
-        <condition>
-            <not>
-                <available file="@{destdir}/index-all.html" />
-            </not>
-        </condition>
-      </fail>
-    </sequential>
-  </macrodef>
+    <macrodef name="create-javadoc">
+      <attribute name="destdir"/>
+      <element name="filesets"/>
+      <sequential>
+        <javadoc destdir="@{destdir}" author="true" version="true" use="true"
+          windowtitle="${ant.project.name} API" classpathref="cassandra.classpath"
+          bottom="Copyright &amp;copy; 2009- The Apache Software Foundation"
+          useexternalfile="yes" encoding="UTF-8" failonerror="false"
+          maxmemory="256m" additionalparam="${jdk11plus-javac-exports}">
+          <filesets/>
+        </javadoc>
+        <fail message="javadoc failed">
+          <condition>
+              <not>
+                  <available file="@{destdir}/index-all.html" />
+              </not>
+          </condition>
+        </fail>
+      </sequential>
+    </macrodef>
+
+    <!-- filterset to apply to pom templates -->
+    <filterset id="pom-template">
+        <filter token="version" value="${version}"/>
+        <filter token="final.name" value="${final.name}"/>
+        <filter token="jamm.version" value="${jamm.version}"/>
+        <filter token="allocation-instrumenter.version" value="${allocation-instrumenter.version}"/>
+        <filter token="ecj.version" value="${ecj.version}"/>
+        <filter token="asm.version" value="${asm.version}"/>
+        <filter token="jacoco.version" value="${jacoco.version}"/>
+        <filter token="jflex.version" value="${jflex.version}"/>
+    </filterset>

Review Comment:
   moved from write-poms as is used in sstableloader-write-pom



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041158276


##########
build.xml:
##########
@@ -2098,6 +2106,16 @@
             file="${build.dir}/${final.name}-javadoc.jar"
             classifier="javadoc"/>
 
+    <!-- the cassandra-sstableloader jar -->
+    <deploy pomFile="${build.dir}/${final.name}-sstableloader.pom"
+            file="${build.dir}/tools/lib/sstableloader.jar"/>
+    <deploy pomFile="${build.dir}/${final.name}-sstableloader.pom"
+            file="${build.dir}/${final.name}-sstableloader-sources.jar"
+            classifier="sources"/>
+    <deploy pomFile="${build.dir}/${final.name}-sstableloader.pom"
+            file="${build.dir}/${final.name}-sstableloader-javadoc.jar"
+            classifier="javadoc"/>
+

Review Comment:
   This is a bit of compromise as I figure `publish` is called explicitly on release, and I figure it likely that one would want to publish all artifacts always.  Alternatively we could use `antcall` and make a `publish-sstableloader` task, but it would redundantly run tasks again.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041162082


##########
.build/sstableloader-deps-template.xml:
##########


Review Comment:
   Changed this, including renaming the other 3 pom files, in 12e681a :+1:



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2876461115

   Sounds good, I figured we'd want to rebase/squash before merge anyways but didn't want to mess with the history in middle of review.  Will take care of that now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#issuecomment-2889112957

   Thanks for running tests and reviewing!  I'll fix up the changelog and merge this shortly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041163031


##########
tools/bin/sstableloader:
##########
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+if [ "x$CASSANDRA_INCLUDE" = "x" ]; then

Review Comment:
   resolving conversation as in other comments noted that was able to make both the debian and redhat builds work :+1:



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2041163356


##########
ide/nbproject/project.xml:
##########
@@ -35,6 +35,12 @@
                     <location>${project.dir}/tools/fqltool/src</location>
                     <encoding>UTF-8</encoding>
                 </source-folder>
+                <source-folder>

Review Comment:
   Ah, forgot to note here, I did verify this works in netbeans (screenshot in pr description)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998583185


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   hmmm thinking about it more, I see what we are doing here. This script is still part of tarball, it is just not "installed" as part of deb / rpm. When I look closely, I see this was installed to `/usr/bin/sstableloader` and `tools/bin/sstableloader` can not be installed there as well upon installing tools package because it would replace what was already in `/usr/bin/sstableloader`. Huh.
   
   But still. I don't think this is a _packaging issue_. Because e.g. a developer might still execute this script as he develops. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998994076


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \

Review Comment:
   changed the error message and removed the else path :+1:



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998992370


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   actually, we still need the existing code because we need to resolve `$CASSANDRA_HOME`.  Alternatively we could just imply the tools directory is in /tools/bin, but knowing that maybe can just keep it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2000221774


##########
build.xml:
##########
@@ -2002,18 +2094,26 @@
   <classpathentry kind="src" output="${test.classes}" path="test/memory"/>
   <classpathentry kind="src" output="${test.classes}" path="test/burn"/>
   <classpathentry kind="src" output="${test.classes}" path="test/anttasks"/>
+  <classpathentry kind="src" output="${test.classes}" path="test/harry/main" />
   <classpathentry kind="src" output="${test.classes}" path="test/distributed"/>
   <classpathentry kind="src" output="${test.classes}" path="test/simulator/asm"/>
   <classpathentry kind="src" output="${test.classes}" path="test/simulator/main"/>
   <classpathentry kind="src" output="${test.classes}" path="test/resources" />
   <classpathentry kind="src" path="tools/stress/src"/>
   <classpathentry kind="src" path="tools/fqltool/src"/>
+  <classpathentry kind="src" path="tools/sstableloader/src"/>
   <classpathentry kind="src" output="build/test/stress-classes" path="tools/stress/test/unit" />
   <classpathentry kind="src" output="build/test/fqltool-classes" path="tools/fqltool/test/unit" />
-  <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
+  <classpathentry kind="src" output="build/test/sstableloader-classes" path="tools/sstableloader/test/unit" />
+  <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-11">
+      <attributes>
+        <attribute name="module" value="true"/>
+          <attribute name="add-exports" value="java.base/jdk.internal.misc=ALL-UNNAMED:java.base/jdk.internal.ref=ALL-UNNAMED:java.base/jdk.internal.util=ALL-UNNAMED:java.base/sun.nio.ch=ALL-UNNAMED:java.management.rmi/com.sun.jmx.remote.internal.rmi=ALL-UNNAMED:java.rmi/sun.rmi.registry=ALL-UNNAMED:java.rmi/sun.rmi.server=ALL-UNNAMED:java.rmi/sun.rmi.transport.tcp=ALL-UNNAMED:java.sql/java.sql=ALL-UNNAMED"/>
+	  <attribute name="add-opens" value="java.base/java.lang.module=ALL-UNNAMED:java.base/java.net=ALL-UNNAMED:java.base/jdk.internal.loader=ALL-UNNAMED:java.base/jdk.internal.ref=ALL-UNNAMED:java.base/jdk.internal.reflect=ALL-UNNAMED:java.base/jdk.internal.math=ALL-UNNAMED:java.base/jdk.internal.module=ALL-UNNAMED:java.base/jdk.internal.util.jar=ALL-UNNAMED:jdk.management/com.sun.management.internal=ALL-UNNAMED"/>
+      </attributes>
+  </classpathentry>
   <classpathentry kind="output" path="build/classes/eclipse"/>
   <classpathentry kind="lib" path="test/conf"/>
-  <classpathentry kind="lib" path="${java.home}/../lib/tools.jar"/>

Review Comment:
   tools.jar was removed in [JEP-220](https://openjdk.org/jeps/220) (JDK9): 
   
   > Removed: rt.jar and tools.jar
   
   > The class and resource files previously stored in lib/rt.jar, lib/tools.jar, lib/dt.jar, and various other internal JAR files are now stored in a more efficient format in implementation-specific files in the lib directory. The format of these files is not specified and is subject to change without notice.
   
   Eclipse no longer seems to need this and was failing to build otherwise.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2000224865


##########
build.xml:
##########
@@ -2002,18 +2094,26 @@
   <classpathentry kind="src" output="${test.classes}" path="test/memory"/>
   <classpathentry kind="src" output="${test.classes}" path="test/burn"/>
   <classpathentry kind="src" output="${test.classes}" path="test/anttasks"/>
+  <classpathentry kind="src" output="${test.classes}" path="test/harry/main" />
   <classpathentry kind="src" output="${test.classes}" path="test/distributed"/>
   <classpathentry kind="src" output="${test.classes}" path="test/simulator/asm"/>
   <classpathentry kind="src" output="${test.classes}" path="test/simulator/main"/>
   <classpathentry kind="src" output="${test.classes}" path="test/resources" />
   <classpathentry kind="src" path="tools/stress/src"/>
   <classpathentry kind="src" path="tools/fqltool/src"/>
+  <classpathentry kind="src" path="tools/sstableloader/src"/>
   <classpathentry kind="src" output="build/test/stress-classes" path="tools/stress/test/unit" />
   <classpathentry kind="src" output="build/test/fqltool-classes" path="tools/fqltool/test/unit" />
-  <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
+  <classpathentry kind="src" output="build/test/sstableloader-classes" path="tools/sstableloader/test/unit" />
+  <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-11">

Review Comment:
   I opted to configure for Java 11 here as figure it's best to work off the minimal supported version.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r2000226951


##########
build.xml:
##########
@@ -1986,7 +2078,7 @@
     </path>
     <pathconvert property="eclipse-libs-list" refid="eclipse-project-libs-path" pathsep="${line.separator}">
         <mapper>
-            <regexpmapper from="^(.*)$$" to='&lt;classpathentry kind="lib" path="\1\" \/&gt;'/>
+            <regexpmapper from="^(.*)$$" to='  &lt;classpathentry kind="lib" path="\1" /&gt;'/>

Review Comment:
   This was previously generating invalid xml in the .classpath file:
   
   ```
   <classpathentry kind="lib" path="/home/tolbertam/Documents/Projects/cassandra/lib/ST4-4.0.8.jar\" \/>
   ```
   
   In particular the `\/>` part is invalid but I expect `...jar\"` probably would throw things off too.  Also added two whitespaces to the beginning to make the spacing look right (not reqiuired, just prettier). 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998551372


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then
-    echo "You must set the CLASSPATH var" >&2
-    exit 1
-fi
+SSTABLELOADER_PATH="$CASSANDRA_HOME/tools/bin/sstableloader"
 
-if [ "x$MAX_HEAP_SIZE" = "x" ]; then
-    MAX_HEAP_SIZE="256M"
+if [ ! -f "$SSTABLELOADER_PATH" ]; then
+    echo "Note: sstableloader has moved to the tools directory. \
+Detected that $SSTABLELOADER_PATH does not exist, this is likely a packaging \
+issue as this script should not be installed." >&2
+    exit 1
+else
+    echo "Warning: sstableloader has been moved to the tools directory. \

Review Comment:
   @tolbertam I am not sure we want to do this. The logic should be "if the script in tool does not exist, tell user that it moved to tools and they should install that package, otherwise run it".
   
   In other words, there is no reason to emit a warning telling a user that script moved and then execute that script. Why should a user care it moved if it works? I would just remove that "else" branch completely.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-20328: Migrate sstableloader code to its own tools directory and artifact [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic commented on code in PR #3899:
URL: https://github.com/apache/cassandra/pull/3899#discussion_r1998547027


##########
bin/sstableloader:
##########
@@ -32,18 +35,18 @@ elif [ -r "$CASSANDRA_INCLUDE" ]; then
     . "$CASSANDRA_INCLUDE"
 fi
 
-if [ -z "$CLASSPATH" ]; then

Review Comment:
   @tolbertam why have you removed all of this?
   
   ```
   if [ -z "$CLASSPATH" ]; then
       echo "You must set the CLASSPATH var" >&2
       exit 1
   fi
   
   if [ "x$MAX_HEAP_SIZE" = "x" ]; then
       MAX_HEAP_SIZE="256M"
   fi
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org