[jira] [Commented] (HADOOP-16336) finish variable is unused in ZStandardCompressor

["xuzq (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898792#comment-16898792 ] 

xuzq commented on HADOOP-16336:
-------------------------------

I'm so sorry to discuss this issue again.

I think the  {{finish}} variable is useful, and it can end one frame in ZSTD.

If we disable the finish variable, there will be only one frame in ZSTD.
{code:java}
JNIEXPORT jint Java_org_apache_hadoop_io_compress_zstd_ZStandardCompressor_deflateBytesDirect
(JNIEnv *env, jobject this, jobject uncompressed_direct_buf, jint uncompressed_direct_buf_off, jint uncompressed_direct_buf_len, jobject compressed_direct_buf, jint compressed_direct_buf_len ) {
    ZSTD_CStream* const stream = (ZSTD_CStream*) (*env)->GetLongField(env, this, ZStandardCompressor_stream);
    if (!stream) {
        THROW(env, "java/lang/NullPointerException", NULL);
        return (jint)0;
    }

    jlong bytes_read = (*env)->GetLongField(env, this, ZStandardCompressor_bytesRead);
    jlong bytes_written = (*env)->GetLongField(env, this, ZStandardCompressor_bytesWritten);
    jboolean finish = (*env)->GetBooleanField(env, this, ZStandardCompressor_finish);

    // Get the input direct buffer
    void * uncompressed_bytes = (*env)->GetDirectBufferAddress(env, uncompressed_direct_buf);
    if (!uncompressed_bytes) {
        THROW(env, "java/lang/InternalError", "Undefined memory address for uncompressedDirectBuf");
        return (jint) 0;
    }

    // Get the output direct buffer
    void * compressed_bytes = (*env)->GetDirectBufferAddress(env, compressed_direct_buf);
    if (!compressed_bytes) {
        THROW(env, "java/lang/InternalError", "Undefined memory address for compressedDirectBuf");
        return (jint) 0;
    }

    ZSTD_inBuffer input = { uncompressed_bytes, uncompressed_direct_buf_len, uncompressed_direct_buf_off };
    ZSTD_outBuffer output = { compressed_bytes, compressed_direct_buf_len, 0 };

    size_t size;
    if (uncompressed_direct_buf_len != 0) {
        size = dlsym_ZSTD_compressStream(stream, &output, &input);
        if (dlsym_ZSTD_isError(size)) {
            THROW(env, "java/lang/InternalError", dlsym_ZSTD_getErrorName(size));
            return (jint) 0;
        }
    }
    if (finish && input.pos == input.size) {
        // end the stream, flush and  write the frame epilogue
        size = dlsym_ZSTD_endStream(stream, &output);
        if (!size) {
            (*env)->SetBooleanField(env, this, ZStandardCompressor_finished, JNI_TRUE);
        }
    } else {
        // need to flush the output buffer
        // this also updates the output buffer position.
        size = dlsym_ZSTD_flushStream(stream, &output);
    }
    if (dlsym_ZSTD_isError(size)) {
        THROW(env, "java/lang/InternalError", dlsym_ZSTD_getErrorName(size));
        return (jint) 0;
    }

    bytes_read += input.pos;
    bytes_written += output.pos;
    (*env)->SetLongField(env, this, ZStandardCompressor_bytesRead, bytes_read);
    (*env)->SetLongField(env, this, ZStandardCompressor_bytesWritten, bytes_written);

    (*env)->SetIntField(env, this, ZStandardCompressor_uncompressedDirectBufOff, input.pos);
    (*env)->SetIntField(env, this, ZStandardCompressor_uncompressedDirectBufLen, input.size - input.pos);
    return (jint) output.pos;
}
{code}

> finish variable is unused in ZStandardCompressor
> ------------------------------------------------
>
>                 Key: HADOOP-16336
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16336
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 3.3.0
>            Reporter: Daniel Templeton
>            Priority: Trivial
>              Labels: newbie
>             Fix For: 3.3.0
>
>
> The boolean {{finish}} variable is unused and can be removed:
> {code:java}
>   private boolean finish, finished;
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org