You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by GitBox <gi...@apache.org> on 2021/12/20 11:54:40 UTC
[GitHub] [incubator-nuttx] anchao opened a new pull request #5039: libc/lzfcompress: add lzf compress stream
anchao opened a new pull request #5039:
URL: https://github.com/apache/incubator-nuttx/pull/5039
## Summary
libc/lzfcompress: add lzf compress stream
compress stream based on lzf algorithm:
struct lib_rawoutstream_s rawstream;
struct lib_lzfoutstream_s lzfstream;
lib_rawoutstream(&rawstream, fd);
lib_lzfoutstream(&lzfstream, &rawstream);
Signed-off-by: chao.an <an...@xiaomi.com>
## Impact
N/A, new feature
## Testing
write the compress stream to file descriptor
```
struct lib_rawoutstream_s rawstream;
struct lib_lzfoutstream_s lzfstream;
lib_rawoutstream(&rawstream, fd);
lib_lzfoutstream(&lzfstream, &rawstream);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on a change in pull request #5039: libc/lzfcompress: add lzf compress stream
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on a change in pull request #5039:
URL: https://github.com/apache/incubator-nuttx/pull/5039#discussion_r772356059
##########
File path: include/nuttx/streams.h
##########
@@ -173,6 +175,21 @@ struct lib_rawsostream_s
int fd;
};
+/* LZF compressed stream pipeline */
+
+#ifdef CONFIG_LIBC_LZF
+struct lib_lzfoutstream_s
+{
+ struct lib_outstream_s public;
+ FAR struct lib_outstream_s *backend;
+ lzf_state_t state;
+ size_t offset;
+ char in[LZF_MAX_HDR_SIZE +
Review comment:
move LZF_MAX_HDR_SIZE to header
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on a change in pull request #5039: libc/lzfcompress: add lzf compress stream
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on a change in pull request #5039:
URL: https://github.com/apache/incubator-nuttx/pull/5039#discussion_r772355857
##########
File path: libs/libc/stream/lib_lzfcompress.c
##########
@@ -0,0 +1,143 @@
+/****************************************************************************
+ * libs/libc/stream/lib_lzfcompress.c
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership. The
+ * ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
+ * License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+ ****************************************************************************/
+
+/****************************************************************************
+ * Included Files
+ ****************************************************************************/
+
+#include <nuttx/config.h>
+
+#include <unistd.h>
+#include <nuttx/streams.h>
+
+/****************************************************************************
+ * Private Functions
+ ****************************************************************************/
+
+/****************************************************************************
+ * Name: lzfoutstream_flush
+ ****************************************************************************/
+
+static int lzfoutstream_flush(FAR struct lib_outstream_s *this)
+{
+ FAR struct lib_lzfoutstream_s *stream =
+ (FAR struct lib_lzfoutstream_s *)this;
+ FAR struct lzf_header_s *header;
+ size_t outlen;
+
+ if (stream->offset > 0)
+ {
+ outlen = lzf_compress(stream->in, stream->offset,
+ &stream->out[LZF_MAX_HDR_SIZE],
+ stream->offset, stream->state, &header);
+ if (outlen > 0)
+ {
+ stream->backend->puts(stream->backend, header, outlen);
+ }
+
+ stream->offset = 0;
+ }
+
+ return stream->backend->flush(stream->backend);
+}
+
+/****************************************************************************
+ * Name: lzfoutstream_puts
+ ****************************************************************************/
+
+static int lzfoutstream_puts(FAR struct lib_outstream_s *this,
+ FAR const void *buf, int len)
+{
+ FAR struct lib_lzfoutstream_s *stream =
+ (FAR struct lib_lzfoutstream_s *)this;
+ FAR struct lzf_header_s *header;
+ FAR const char *ptr = buf;
+ size_t total = len;
+ size_t copyin;
+ size_t outlen;
+ int ret;
+
+ while (total > 0)
+ {
+ copyin = stream->offset + total > CONFIG_STREAM_LZF_BLOG ?
Review comment:
it's better to define STREAM_LZF_SIZE equals (1 << CONFIG_STREAM_LZF_BLOG)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on a change in pull request #5039: libc/lzfcompress: add lzf compress stream
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on a change in pull request #5039:
URL: https://github.com/apache/incubator-nuttx/pull/5039#discussion_r772334856
##########
File path: include/nuttx/streams.h
##########
@@ -173,6 +175,20 @@ struct lib_rawsostream_s
int fd;
};
+/* LZF compressed stream pipeline */
+
+#ifdef CONFIG_LIBC_LZF
+struct lib_lzfoutstream_s
+{
+ struct lib_outstream_s public;
+ FAR struct lib_outstream_s *backend;
+ lzf_state_t state;
+ size_t offset;
+ FAR char buffer[2][CONFIG_STREAM_LZF_BLOG +
Review comment:
1 << CONFIG_STREAM_LZF_BLOG
##########
File path: libs/libc/stream/lib_lzfcompress.c
##########
@@ -0,0 +1,149 @@
+/****************************************************************************
+ * libs/libc/stream/lib_lzfcompress.c
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership. The
+ * ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
+ * License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+ ****************************************************************************/
+
+/****************************************************************************
+ * Included Files
+ ****************************************************************************/
+
+#include <nuttx/config.h>
+
+#include <unistd.h>
+#include <nuttx/streams.h>
+
+/****************************************************************************
+ * Private Functions
+ ****************************************************************************/
+
+/****************************************************************************
+ * Name: lzfoutstream_flush
+ ****************************************************************************/
+
+static int lzfoutstream_flush(FAR struct lib_outstream_s *this)
+{
+ FAR struct lib_lzfoutstream_s *stream =
+ (FAR struct lib_lzfoutstream_s *)this;
+ FAR char *out = stream->buffer[1];
+ FAR char *in = stream->buffer[0];
+ FAR struct lzf_header_s *header;
+ size_t outlen;
+
+ if (stream->offset > 0)
+ {
+ outlen = lzf_compress(&in[LZF_MAX_HDR_SIZE],
+ stream->offset, &out[LZF_MAX_HDR_SIZE],
+ stream->offset > 4 ?
+ stream->offset - 4 : stream->offset,
+ stream->state, &header);
+ if (outlen > 0)
+ {
+ stream->backend->puts(stream->backend, header, outlen);
+ }
+
+ stream->offset = 0;
+ }
+
+ return stream->backend->flush(stream->backend);
+}
+
+/****************************************************************************
+ * Name: lzfoutstream_puts
+ ****************************************************************************/
+
+static int lzfoutstream_puts(FAR struct lib_outstream_s *this,
+ FAR const void *buf, int len)
+{
+ FAR struct lib_lzfoutstream_s *stream =
+ (FAR struct lib_lzfoutstream_s *)this;
+ FAR char *out = stream->buffer[1];
+ FAR char *in = stream->buffer[0];
+ FAR struct lzf_header_s *header;
+ FAR const char *ptr = buf;
+ size_t total = len;
+ size_t copyin;
+ size_t outlen;
+ int ret;
+
+ while (total > 0)
+ {
+ copyin = stream->offset + total > CONFIG_STREAM_LZF_BLOG ?
+ CONFIG_STREAM_LZF_BLOG - stream->offset : total;
+
+ memcpy(in + LZF_MAX_HDR_SIZE + stream->offset, ptr, copyin);
+
+ ptr += copyin;
+ stream->offset += copyin;
+ this->nput += copyin;
+ total -= copyin;
+
+ if (stream->offset == CONFIG_STREAM_LZF_BLOG)
+ {
+ outlen = lzf_compress(&in[LZF_MAX_HDR_SIZE], stream->offset,
Review comment:
why not use the first LZF_MAX_HDR_SIZE of in buffer
##########
File path: include/nuttx/streams.h
##########
@@ -173,6 +175,20 @@ struct lib_rawsostream_s
int fd;
};
+/* LZF compressed stream pipeline */
+
+#ifdef CONFIG_LIBC_LZF
+struct lib_lzfoutstream_s
+{
+ struct lib_outstream_s public;
+ FAR struct lib_outstream_s *backend;
+ lzf_state_t state;
+ size_t offset;
+ FAR char buffer[2][CONFIG_STREAM_LZF_BLOG +
Review comment:
remove FAR and split buffer to in and out
##########
File path: libs/libc/stream/lib_lzfcompress.c
##########
@@ -0,0 +1,149 @@
+/****************************************************************************
+ * libs/libc/stream/lib_lzfcompress.c
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership. The
+ * ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
+ * License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+ ****************************************************************************/
+
+/****************************************************************************
+ * Included Files
+ ****************************************************************************/
+
+#include <nuttx/config.h>
+
+#include <unistd.h>
+#include <nuttx/streams.h>
+
+/****************************************************************************
+ * Private Functions
+ ****************************************************************************/
+
+/****************************************************************************
+ * Name: lzfoutstream_flush
+ ****************************************************************************/
+
+static int lzfoutstream_flush(FAR struct lib_outstream_s *this)
+{
+ FAR struct lib_lzfoutstream_s *stream =
+ (FAR struct lib_lzfoutstream_s *)this;
+ FAR char *out = stream->buffer[1];
+ FAR char *in = stream->buffer[0];
+ FAR struct lzf_header_s *header;
+ size_t outlen;
+
+ if (stream->offset > 0)
+ {
+ outlen = lzf_compress(&in[LZF_MAX_HDR_SIZE],
+ stream->offset, &out[LZF_MAX_HDR_SIZE],
+ stream->offset > 4 ?
+ stream->offset - 4 : stream->offset,
+ stream->state, &header);
+ if (outlen > 0)
+ {
+ stream->backend->puts(stream->backend, header, outlen);
+ }
+
+ stream->offset = 0;
+ }
+
+ return stream->backend->flush(stream->backend);
+}
+
+/****************************************************************************
+ * Name: lzfoutstream_puts
+ ****************************************************************************/
+
+static int lzfoutstream_puts(FAR struct lib_outstream_s *this,
+ FAR const void *buf, int len)
+{
+ FAR struct lib_lzfoutstream_s *stream =
+ (FAR struct lib_lzfoutstream_s *)this;
+ FAR char *out = stream->buffer[1];
+ FAR char *in = stream->buffer[0];
+ FAR struct lzf_header_s *header;
+ FAR const char *ptr = buf;
+ size_t total = len;
+ size_t copyin;
+ size_t outlen;
+ int ret;
+
+ while (total > 0)
+ {
+ copyin = stream->offset + total > CONFIG_STREAM_LZF_BLOG ?
+ CONFIG_STREAM_LZF_BLOG - stream->offset : total;
+
+ memcpy(in + LZF_MAX_HDR_SIZE + stream->offset, ptr, copyin);
+
+ ptr += copyin;
+ stream->offset += copyin;
+ this->nput += copyin;
+ total -= copyin;
+
+ if (stream->offset == CONFIG_STREAM_LZF_BLOG)
+ {
+ outlen = lzf_compress(&in[LZF_MAX_HDR_SIZE], stream->offset,
+ &out[LZF_MAX_HDR_SIZE],
+ stream->offset - 4, stream->state,
Review comment:
should we remove -4?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 merged pull request #5039: libc/lzfcompress: add lzf compress stream
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 merged pull request #5039:
URL: https://github.com/apache/incubator-nuttx/pull/5039
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org