You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@avro.apache.org by ie...@apache.org on 2021/06/11 14:59:55 UTC
[avro] branch master updated (6dc6055 -> 5fd979f)
This is an automated email from the ASF dual-hosted git repository.
iemejia pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/avro.git.
from 6dc6055 AVRO-2921: Type Hints for avro.schema (#1252)
add 7741c90 Initial commit
add fe7838a Add LICENSE
add 797622f Basic implementation
add 5931c2a Harmonize named types into a Name struct
add fa255e0 Convert data, Encode data
add f1220e8 Add ToAvro for () and HashMaps
add 221c5eb Introduce HasSchema trait
add 22fbfcf Add Writer implementation
add e66a1a6 Add schema json serialization
add 54ee76c Correclty encode types implementing ToAvro
add 8dcefe7 Use mem::transmute to encode floating-point numbers
add 9429baf Consider header written only on success
add 0ece0eb Use usize for Fixed size
add 40abec8 Make Schema implement serde::ser::Serialize
add 8340007 Add deflate and snappy codecs
add e34c2d4 Improve zigzag functions naming
add 523eda8 Make Writer own its inner writer
add d78d679 Add Schema::parse_str
add cc5eea1 More code cleanup
add c2ee646 Perform schema-related conversion in EncodeAvro
add 259f6a4 Make schema optional for record
add 0a6d68b Add basic decoding
add a0e36c2 Use VecDeque instead of Vec when decoding
add b62c916 Move codec into src/codec.rs
add 8a17afe Allow decompression with codecs
add 02ad501 Remove serde_json from example
add 62ac563 Check markers after each block
add 279a6fb Add serde::Serializer code skeleton
add b6cb9b6 Implement Serializer
add 4418664 Attempt to impl Serialize for Value
add 3ea6c20 Implement Deserializer
add e09f5d6 Improve folder hierarchy
add f22501f Make use of forward_to_deserialize_any!
add 26815f5 Add schema resolution and use it while deserializing
add 85bc4e3 Implement Serialize for types::Value and types::Record
add 7490170 Use default when serializing if no value is found.
add cf3bf1a Add helper methods to interact with Value and Record
add 571076f Flatten architecture
add e2e61dd Only support valid Value encoding
add 9dc1db0 Remove non-stable slice_pattern feature dependency
add 0a0a66a Let Writer only interact with Value
add 90cff7e WIP
add 402180d WIP
add ef930cc Remove comments
add 65e2658 Refactor
add 229830b Add StructDeserializer
add 8d008e5 Only validate schema when writing
add 4185526 Support schema resolution through reader_schema
add 2ecd3aa Refactor record_schema to rschema
add 79c3bf5 Support Enum serialization
add 8135e24 Avoid computing record schema lookup at every .put
add 7dada20 Refactor to remove warnings
add df4e8be Fix tests
add b2ca48b Make schema compliant with avro recursive primitive types
add c2c9f25 Do not create intermediate HashMap when writing records
add 22f7ecf Buffer reads/writes
add c87fe92 Add benchmark
add 662faf9 Add prototype for Value::from_json_value
add 1004141 Hackathon release
add df94ea7 Allow Value::Map to be resolved into a Record
add 5932ae7 Add Long/Double resolving into Int/Float
add 5554c4a Avoid non-needed allocations when encoding
add 3dfff91 Some quality of life improvements (#1)
add 1c22ea7 Move hello example to doc test, add rust benchmark based on benchmark example and add travis configuration (#2)
add f102828 Add SingleWriter avro writer (#3)
add b77f77c Add {reader,writer}_schema to Reader
add db89414 Add docs and tests to codec module and make private modules that don't need to be public (#4)
add e881e83 Full Enum support and reader's iterator returns Result (#15)
add 39dd1ee Add documentation and unit tests to types.rs
add cc55d86 Writer module overhauling (#20)
add 738fa3b Reader module tests, documentation and interface overhaul (#21)
add d6e9c79 Schema documentation and unit tests. (#22)
add deca059 Avoid calling .resolve when writing datum (#23)
add 69b1023 Write tutorial and README (#24)
add c0c7a34 Make Cargo.toml crates.io-compliant
add faadbc4 Use an available package name
add aee5164 Rename crate from avro to avro-rs
add 2baa511 Bump version to 0.1.1
add 5dde3b5 Various fixes/improvements to the documentation (#26)
add b4138d0 Bump version to 0.2.0
add 90a6498 Use crates.io badge (#28)
add f1434f0 Move from string as errors to custom fail types (#29)
add 520d2ce Use a single &[u8] when decoding items in Reader (#32)
add ef8878d Bump version to 0.3.0
add 04f82c2 Encode/decode Union values with a leading zig-zag long
add c3a19a2 Bump version to 0.3.1
add c94cbb0 Added some missing serialization fields for `Schema::Record`
add d39cc23 Bump version to 0.3.2
add 372b548 Performance Enhancements
add 8b0df4c Remove pub from private methods of Block
add 2660a8b Bump version to 0.4.0
add 6a036fc Linting and clipping
add 3b822b6 Bump version to 0.4.1
add d053b29 Run clippy as pre-commit
add 1860c70 Link the other two repos in README
add c7942e4 Add a maximum allocation size when decoding
add 57bc8c6 Add support for Parsing Canonical Form
add 260d189 Add `to_value` to the API (#46)
add e5c6c55 Add full union support
add e35eb56 Fix encoding of empty Array/Map
add cebc498 impl Send+Sync for Schema
add c3a97bb Bump version to 0.6.0
add 9bf2365 Shift 64-bit integers by sixty-three bits
add 1130001 Convert i32 to i64 earlier during serialization
add fe68328 Bump version to 0.6.1
add 227940c Fix snappy codec
add 37ea688 Bump version to 0.6.2
add 35d2108 Convert to Rust 2018 edition
add 10ea827 Add initial integration-style tests
add 2fc8d9d Add fingerprinting generation to library
add 29f318b Bump version to 0.6.3
add 9492d6c Fix variable-length encoding for big i64 numbers
add 1afc3d9 Bump version to 0.6.4
add 7b820c1 Add edition declaration to Cargo.toml (#64)
add aa1cb56 Fix enum type deserialization bug (#73)
add 9e6b18f Allow Array(Int) to be converted to Bytes
add e5ca80b Bump version to 0.6.5
add 06a1b01 Make use of rust 2018 features (#78)
add c4971ac Port tests from avro/test_io.py (#81)
add 3a93d16 Port tests from apache/avro test_schema.py and implement schema equality based on parsing canonical form (#97)
add 652b233 Negative block lengths are not handled (#100)
add af04ed8 Bump to version v0.6.6
add ffede29 Uncomment enum default test and remove unused test import (#102)
add c9f62a1 Fix comment and explicit match
add 3417ccd Re-export {de,ser}::Error (#104)
add 183893e Fix union resolution (#103)
add 7c72554 Bump to version 0.7.0
add 173e4c5 Fix warnings on master (#112)
add 0bcb437 Remove some unsafe (#113)
add 3336bd1 Fix f32 and f64 encoding/decoding (#116)
add 1f8af33 [Conflict Fix] Partial Rust Enum Serialization/Deserialization Support (#110)
add 4e89943 Bumo to version 0.8.0
add f7f03f3 Add support for logical types (#111)
add 9837a4d Make writer block size configurable (#119)
add 3d774b9 Bump version to 0.9.0
add 85d8c45 Port benchmarks to criterion (#114)
add 7a80378 Remove benchmark from travis because they take too long to run[A
add f679ac7 Update changelog
add 83baa43 Configure long running benchmarks
add 4c184f6 Revert "Remove benchmark from travis because they take too long to run[A"
add 1a9d3ae Update travis-test.sh
add 9589175 Compile benchmark in travis instead of running them and do it on stable
add 86b4587 Fix clippy lints and add to CI (#124)
add 5acbfbf Add a test for nullable logical types (#123)
add 477c71e Fix reader len bug (#126)
add 5b7e52e remove not relevant comment
add d7aa0a8 Bump version to 0.9.1
add d043d23 Fix remaining clippy lints (#129)
add f343383 Generate README.md with `cargo-readme` (#131)
add 1988598 Add schema compatibility to library (#88)
add 7395db8 Make writer.into_inner() call flush() and return a Result (#132)
add 606ba8d Bump to version 0.10.0
add e1cffc5 Update CHANGELOG.md
add da3580e Update README.tpl
add 45c941d Sync readme
add c1fc9df Fix broken link markdown
add ce7c710 Install clippy before using it
add df61e8a Bump digest library to 0.9 and its dependencies (#133)
add 3fb71e0 Deprecate ToAvro in favor of From<T> for Value impls (#137)
add 72c0ffc Replace some manual from_str impls with strum (#136)
add 9592038 Move from failure to thiserror (#135)
add 656db36 Move to GH actions (#138)
add d841c04 Add rustfmt.toml and consolidate imports (#143)
add 69807e9 Update ci.yml
add 7a87b53 Move codec to use site (#142)
add d3f3c6e Handle logical types in canonical form schema (#144)
add d719de2 Fix length casting during decoding (#145)
add 95ae8d2 Update README to remove both old travis badges and pointers to unmaintained FFI crates
add 6c3b4cd Update libflate to version 1.0 (#151)
add b3edfac Make errors more fine-grained (#146)
add 743a6df Update migration_guide.md
add 4aae22d support to convert avro value to json value (#155)
add 8249051 Implement deserialize for Uuid (#153)
add a89d82e Update CHANGELOG.md
add 0ca45b4 Bump version to 0.11.0
add c2dbe43 Implement the Rabin fingerprint algorithm (#157)
add 2e24700 Update CHANGELOG.md
add 389e5ea Schedule ci.yml
add 2817b09 Schedule clippy.yml
add 9eebe82 Add workflow_dispatch to all workflows
add f59297f Run and fix docs tests
add 1b3e973 Fix matches macro clippy error
add 9d99073 Bump MSRV to 1.42.0 for subslices patterns and more clippy fixes
add 1e16824 Remove trailing comma from matches! because support added in 1.48.0
add 9a42a2b Revert "Remove trailing comma from matches! because support added in 1.48.0"
add 226ba8a Tired of trying to compile on older versions. MSRV is 1.48.0
add 318cb46 Appease the clippy gods
add 9076770 Update ci.yml
add 4e372de Strip more fields in PCF (#164)
add d5b1970 Update changelog
add d5179c3 Try using stable toolchain for clippy check
add de6153e Bump to v0.12.0
add f28acbb Bump rand crate to 0.7.0 (#168)
add 5d97f28 Remove unnecessary cast (clippy warning) (#172)
add d8299dc allow Value::Bytes to be assigned to Schema::Fixed ... (#171)
add 9c248bd Allow resolution of union schemas with logical types (#176)
add 8c02574 Use `main` as branch name in the README
add eef23fc Support for parsing a list of schemas which may have cross dependencies (#173)
add f0c2ac2 Bump to v0.13.0
add 8e5a8ef Validate maps as records (#178)
add 941000b Validate enum symbols (#181)
add 88a6014 Import avro-rs into Apache Avro
add 7f15e3d Switch LICENSE to Apache Software License v2 (ASL)
add f84c53c Add Apache Software License (ASL) headers
add e80bd25 Add lang/rust to the RAT configuration
add 8cf8e70 Integrate Rust module into parent Github Actions
new b394458 Address Rust PR review comments
new 5fd979f AVRO-3149: Add Rust based Implementation of Avro
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/labeler.yml | 1 +
.../{labeler.yml => test-lang-rust-audit.yml} | 28 +-
.github/workflows/test-lang-rust-ci.yml | 75 ++
.../{labeler.yml => test-lang-rust-clippy.yml} | 31 +-
doc/forrest.properties => lang/rust/.activate.sh | 10 +-
doc/forrest.properties => lang/rust/.deactivate.sh | 12 +-
lang/rust/.gitignore | 7 +
.../rust/.pre-commit-config.yaml | 28 +-
lang/rust/.requirements-precommit.txt | 1 +
lang/rust/CHANGELOG.md | 163 +++
lang/rust/Cargo.toml | 72 ++
lang/{js => rust}/LICENSE | 0
lang/rust/Makefile | 105 ++
lang/rust/README.md | 626 +++++++++
lang/rust/README.tpl | 20 +
lang/rust/benches/quickstop-null.avro | Bin 0 -> 166053 bytes
lang/rust/benches/serde.rs | 338 +++++
lang/rust/benches/serde_json.rs | 79 ++
lang/rust/benches/single.rs | 193 +++
lang/rust/examples/benchmark.rs | 138 ++
lang/rust/examples/to_value.rs | 29 +
lang/rust/migration_guide.md | 107 ++
doc/forrest.properties => lang/rust/rustfmt.toml | 9 +-
lang/rust/src/codec.rs | 177 +++
lang/rust/src/de.rs | 899 +++++++++++++
lang/rust/src/decimal.rs | 74 ++
lang/rust/src/decode.rs | 315 +++++
lang/rust/src/duration.rs | 155 +++
lang/rust/src/encode.rs | 188 +++
lang/rust/src/error.rs | 378 ++++++
lang/rust/src/lib.rs | 976 ++++++++++++++
lang/rust/src/rabin.rs | 154 +++
lang/rust/src/reader.rs | 435 +++++++
lang/rust/src/schema.rs | 1367 ++++++++++++++++++++
lang/rust/src/schema_compatibility.rs | 916 +++++++++++++
lang/rust/src/ser.rs | 999 ++++++++++++++
lang/rust/src/types.rs | 1267 ++++++++++++++++++
lang/rust/src/util.rs | 218 ++++
lang/rust/src/writer.rs | 813 ++++++++++++
lang/rust/tests/io.rs | 322 +++++
lang/rust/tests/schema.rs | 1197 +++++++++++++++++
pom.xml | 5 +
42 files changed, 12879 insertions(+), 48 deletions(-)
copy .github/workflows/{labeler.yml => test-lang-rust-audit.yml} (65%)
create mode 100644 .github/workflows/test-lang-rust-ci.yml
copy .github/workflows/{labeler.yml => test-lang-rust-clippy.yml} (59%)
copy doc/forrest.properties => lang/rust/.activate.sh (86%)
copy doc/forrest.properties => lang/rust/.deactivate.sh (86%)
create mode 100644 lang/rust/.gitignore
copy .github/workflows/labeler.yml => lang/rust/.pre-commit-config.yaml (55%)
create mode 100644 lang/rust/.requirements-precommit.txt
create mode 100644 lang/rust/CHANGELOG.md
create mode 100644 lang/rust/Cargo.toml
copy lang/{js => rust}/LICENSE (100%)
create mode 100644 lang/rust/Makefile
create mode 100644 lang/rust/README.md
create mode 100644 lang/rust/README.tpl
create mode 100644 lang/rust/benches/quickstop-null.avro
create mode 100644 lang/rust/benches/serde.rs
create mode 100644 lang/rust/benches/serde_json.rs
create mode 100644 lang/rust/benches/single.rs
create mode 100644 lang/rust/examples/benchmark.rs
create mode 100644 lang/rust/examples/to_value.rs
create mode 100644 lang/rust/migration_guide.md
copy doc/forrest.properties => lang/rust/rustfmt.toml (86%)
create mode 100644 lang/rust/src/codec.rs
create mode 100644 lang/rust/src/de.rs
create mode 100644 lang/rust/src/decimal.rs
create mode 100644 lang/rust/src/decode.rs
create mode 100644 lang/rust/src/duration.rs
create mode 100644 lang/rust/src/encode.rs
create mode 100644 lang/rust/src/error.rs
create mode 100644 lang/rust/src/lib.rs
create mode 100644 lang/rust/src/rabin.rs
create mode 100644 lang/rust/src/reader.rs
create mode 100644 lang/rust/src/schema.rs
create mode 100644 lang/rust/src/schema_compatibility.rs
create mode 100644 lang/rust/src/ser.rs
create mode 100644 lang/rust/src/types.rs
create mode 100644 lang/rust/src/util.rs
create mode 100644 lang/rust/src/writer.rs
create mode 100644 lang/rust/tests/io.rs
create mode 100644 lang/rust/tests/schema.rs
[avro] 01/02: Address Rust PR review comments
Posted by ie...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
iemejia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/avro.git
commit b3944584e0dfea7c945902bde1d850b346aed4fd
Author: Ismaël Mejía <ie...@gmail.com>
AuthorDate: Fri Jun 11 14:09:50 2021 +0200
Address Rust PR review comments
---
.github/workflows/test-lang-rust-audit.yml | 4 ++--
.github/workflows/test-lang-rust-ci.yml | 13 ++++++-------
.github/workflows/test-lang-rust-clippy.yml | 3 +--
3 files changed, 9 insertions(+), 11 deletions(-)
diff --git a/.github/workflows/test-lang-rust-audit.yml b/.github/workflows/test-lang-rust-audit.yml
index 484f454..33b5113 100644
--- a/.github/workflows/test-lang-rust-audit.yml
+++ b/.github/workflows/test-lang-rust-audit.yml
@@ -15,6 +15,7 @@
# specific language governing permissions and limitations
# under the License.
+name: Rust Security Audit
on:
workflow_dispatch:
push:
@@ -26,14 +27,13 @@ on:
- lang/rust/Cargo.toml
- lang/rust/Cargo.lock
-name: Security Audit
jobs:
audit:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- - name: Audit
+ - name: Rust Audit
uses: actions-rs/audit-check@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/test-lang-rust-ci.yml b/.github/workflows/test-lang-rust-ci.yml
index b4086f1..449f558 100644
--- a/.github/workflows/test-lang-rust-ci.yml
+++ b/.github/workflows/test-lang-rust-ci.yml
@@ -15,6 +15,7 @@
# specific language governing permissions and limitations
# under the License.
+name: Rust Continuous Integration
on:
workflow_dispatch:
push:
@@ -25,8 +26,6 @@ on:
- .github/workflows/test-lang-rust-ci.yml
- lang/rust/**
-name: Continuous Integration
-
jobs:
ci:
runs-on: ubuntu-latest
@@ -42,7 +41,7 @@ jobs:
- name: Checkout
uses: actions/checkout@v2
- - name: Toolchain
+ - name: Rust Toolchain
uses: actions-rs/toolchain@v1
with:
profile: minimal
@@ -50,26 +49,26 @@ jobs:
override: true
components: rustfmt
- - name: Format
+ - name: Rust Format
uses: actions-rs/cargo@v1
with:
command: fmt
args: --all -- --check
- - name: Build
+ - name: Rust Build
uses: actions-rs/cargo@v1
with:
command: build
args: --all-features --all-targets
- - name: Test
+ - name: Rust Test
uses: actions-rs/cargo@v1
with:
command: test
args: --all-features --all-targets
# because of https://github.com/rust-lang/cargo/issues/6669
- - name: Test docs
+ - name: Rust Test docs
uses: actions-rs/cargo@v1
with:
command: test
diff --git a/.github/workflows/test-lang-rust-clippy.yml b/.github/workflows/test-lang-rust-clippy.yml
index d1f2d5c..9ac107b 100644
--- a/.github/workflows/test-lang-rust-clippy.yml
+++ b/.github/workflows/test-lang-rust-clippy.yml
@@ -15,6 +15,7 @@
# specific language governing permissions and limitations
# under the License.
+name: Rust Clippy Check
on:
workflow_dispatch:
push:
@@ -25,8 +26,6 @@ on:
- .github/workflows/test-lang-rust-clippy.yml
- lang/rust/**
-name: Clippy Check
-
jobs:
clippy_check:
runs-on: ubuntu-latest
[avro] 02/02: AVRO-3149: Add Rust based Implementation of Avro
Posted by ie...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
iemejia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/avro.git
commit 5fd979fafe50cb55f0cd8129d45be0e1f15c4015
Merge: 6dc6055 b394458
Author: Ismaël Mejía <ie...@gmail.com>
AuthorDate: Fri Jun 11 16:58:52 2021 +0200
AVRO-3149: Add Rust based Implementation of Avro
.github/labeler.yml | 1 +
.../test-lang-rust-audit.yml} | 36 +-
.github/workflows/test-lang-rust-ci.yml | 75 ++
.../test-lang-rust-clippy.yml} | 39 +-
.github/labeler.yml => lang/rust/.activate.sh | 17 +-
.github/labeler.yml => lang/rust/.deactivate.sh | 19 +-
lang/rust/.gitignore | 7 +
.../rust/.pre-commit-config.yaml | 30 +-
lang/rust/.requirements-precommit.txt | 1 +
lang/rust/CHANGELOG.md | 163 +++
lang/rust/Cargo.toml | 72 ++
lang/rust/LICENSE | 202 +++
lang/rust/Makefile | 105 ++
lang/rust/README.md | 626 +++++++++
lang/rust/README.tpl | 20 +
lang/rust/benches/quickstop-null.avro | Bin 0 -> 166053 bytes
lang/rust/benches/serde.rs | 338 +++++
lang/rust/benches/serde_json.rs | 79 ++
lang/rust/benches/single.rs | 193 +++
lang/rust/examples/benchmark.rs | 138 ++
lang/rust/examples/to_value.rs | 29 +
lang/rust/migration_guide.md | 107 ++
.github/labeler.yml => lang/rust/rustfmt.toml | 16 +-
lang/rust/src/codec.rs | 177 +++
lang/rust/src/de.rs | 899 +++++++++++++
lang/rust/src/decimal.rs | 74 ++
lang/rust/src/decode.rs | 315 +++++
lang/rust/src/duration.rs | 155 +++
lang/rust/src/encode.rs | 188 +++
lang/rust/src/error.rs | 378 ++++++
lang/rust/src/lib.rs | 976 ++++++++++++++
lang/rust/src/rabin.rs | 154 +++
lang/rust/src/reader.rs | 435 +++++++
lang/rust/src/schema.rs | 1367 ++++++++++++++++++++
lang/rust/src/schema_compatibility.rs | 916 +++++++++++++
lang/rust/src/ser.rs | 999 ++++++++++++++
lang/rust/src/types.rs | 1267 ++++++++++++++++++
lang/rust/src/util.rs | 218 ++++
lang/rust/src/writer.rs | 813 ++++++++++++
lang/rust/tests/io.rs | 322 +++++
lang/rust/tests/schema.rs | 1197 +++++++++++++++++
pom.xml | 5 +
42 files changed, 13084 insertions(+), 84 deletions(-)