You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tsfile.apache.org by ja...@apache.org on 2026/04/20 09:52:24 UTC
(tsfile) branch develop updated: Add CLAUDE.md files and update .gitignore (#788)
This is an automated email from the ASF dual-hosted git repository.
JackieTien97 pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/tsfile.git
The following commit(s) were added to refs/heads/develop by this push:
new 5e8a38ae Add CLAUDE.md files and update .gitignore (#788)
5e8a38ae is described below
commit 5e8a38ae97896765854a3188926c91b15f63c5f4
Author: Jackie Tien <ja...@gmail.com>
AuthorDate: Mon Apr 20 17:52:19 2026 +0800
Add CLAUDE.md files and update .gitignore (#788)
---
.gitignore | 6 ++++
CLAUDE.md | 67 ++++++++++++++++++++++++++++++++++
cpp/CLAUDE.md | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
java/CLAUDE.md | 101 +++++++++++++++++++++++++++++++++++++++++++++++++++
python/CLAUDE.md | 104 +++++++++++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 386 insertions(+)
diff --git a/.gitignore b/.gitignore
index f3b2b0a9..d2727756 100644
--- a/.gitignore
+++ b/.gitignore
@@ -46,3 +46,9 @@ cpp/third_party/zlib-1.3.1
build/*
cpp/third_party/zlib-1.3.1/treebuild.xml
cpp/third_party/zlib-1.3.1/zlib-1.3.1/treebuild.xml
+
+# Claude Code
+.claude/settings.local.json
+.claude/todos/
+.claude/worktrees/
+.claude/scheduled_tasks.json
diff --git a/CLAUDE.md b/CLAUDE.md
new file mode 100644
index 00000000..3076a77e
--- /dev/null
+++ b/CLAUDE.md
@@ -0,0 +1,67 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Apache TsFile is a columnar storage file format designed for time series data. The project provides implementations in three languages, each in its own top-level directory. The main branch is `develop`.
+
+TsFile uses a hierarchical storage model: **Page → Chunk → ChunkGroup → File**. Data is organized by device, with each device's measurements stored as individual time series. Two write models exist: **aligned** (all measurements share timestamps) and **non-aligned** (independent timestamps per measurement).
+
+## Repository Structure
+
+| Directory | Language | Build System | Details |
+|-----------|----------|--------------|---------|
+| `java/` | Java | Maven | [java/CLAUDE.md](java/CLAUDE.md) |
+| `cpp/` | C++ | CMake | [cpp/CLAUDE.md](cpp/CLAUDE.md) |
+| `python/` | Python | setuptools + Cython | [python/CLAUDE.md](python/CLAUDE.md) |
+
+The root `pom.xml` orchestrates all three via Maven profiles:
+
+```bash
+./mvnw clean verify -P with-java # Java only
+./mvnw clean verify -P with-cpp # C++ only
+./mvnw clean verify -P with-python # Python (requires C++ built first)
+```
+
+## Cross-Module Dependencies
+
+- **Python depends on C++**: The Python module uses Cython to bind to the C++ shared library (`libtsfile`). C++ must be built before Python (`-P with-python` implies C++ build).
+- **Java is independent**: The Java implementation shares no build-time dependency with C++ or Python.
+- All three implementations read/write the same TsFile binary format.
+
+## Code Formatting
+
+```bash
+./mvnw spotless:apply # Format all languages (Java: Google Java Format, C++: clang-format, Python: Black)
+./mvnw spotless:check # Check formatting without modifying
+```
+
+## License Header
+
+Every new file must include the Apache License 2.0 header at the top. Use the comment style appropriate for the file type (e.g., `<!-- -->` for Markdown, `/* */` for Java/C++, `#` for Python). See any existing file for the exact wording.
+
+## Git Commit
+
+- Do NOT add `Co-Authored-By` trailer to commit messages.
diff --git a/cpp/CLAUDE.md b/cpp/CLAUDE.md
new file mode 100644
index 00000000..00157dd5
--- /dev/null
+++ b/cpp/CLAUDE.md
@@ -0,0 +1,108 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# CLAUDE.md — C++ Module
+
+This file provides guidance to Claude Code (claude.ai/code) when working with the C++ module.
+
+## Build Commands
+
+```bash
+# Build (Release, default)
+bash build.sh
+
+# Build variants
+bash build.sh -t=Debug
+bash build.sh -t=RelWithDebInfo
+bash build.sh -a=ON # Enable AddressSanitizer
+bash build.sh -c=ON # Enable code coverage
+
+# Disable optional compression libraries
+bash build.sh --disable-snappy --disable-lz4 --disable-lzokay --disable-zlib
+
+# Or use CMake directly
+mkdir -p build/Release && cd build/Release
+cmake ../.. -DCMAKE_BUILD_TYPE=Release -DBUILD_TEST=ON
+make -j$(nproc)
+
+# Via Maven from repo root
+./mvnw clean verify -P with-cpp
+
+# Run all tests
+./build/Release/lib/TsFile_Test
+
+# Run specific test suite
+./build/Release/lib/TsFile_Test --gtest_filter=SnappyCompressorTest.*
+```
+
+## Build Options (CMake)
+
+All `ON` by default unless noted:
+
+| Option | Purpose |
+|--------|---------|
+| `BUILD_TEST` | Compile tests (GTest 1.12.1, auto-downloaded) |
+| `ENABLE_ANTLR4` | ANTLR4 parser runtime |
+| `ENABLE_SNAPPY` / `ENABLE_LZ4` / `ENABLE_LZOKAY` / `ENABLE_ZLIB` | Compression libraries |
+| `ENABLE_THREADS` | Multi-threaded read/write via pthreads |
+| `ENABLE_ASAN` | AddressSanitizer (`OFF` by default) |
+| `ENABLE_SIMDE` | SIMD Everywhere (`OFF` by default) |
+
+## Source Structure
+
+```
+cpp/src/
+├── common/ # Core types: Schema, Tablet, DeviceId, TsBlock, allocators, config
+├── compress/ # Compression: Snappy, LZ4, LZOKAY, Zlib (factory pattern)
+├── encoding/ # Encoding: Plain, TS2Diff, Gorilla, Dictionary, RLE, Zigzag, SPRINTZ
+├── file/ # File I/O: TsFileIOReader/Writer, RestorableTsFileIOWriter
+├── reader/ # Read path: TsFileReader, QueryExecutor, filters, result sets
+├── writer/ # Write path: TsFileWriter, TsFileTableWriter, ChunkWriter, PageWriter
+├── parser/ # ANTLR4 path parser (grammars + generated code)
+├── cwrapper/ # C language bindings (used by Python module)
+└── utils/ # Utilities: error codes, date handling, fault injection
+```
+
+## Architecture Notes
+
+- **C++11** standard, targets CMake 3.11+
+- Dual data model: **tree-view** (`TsFileTreeWriter/Reader`) and **table-view** (`TsFileTableWriter`, `TableQueryExecutor`)
+- Parallel column encoding in table write path, controlled by `ENABLE_THREADS`
+- Third-party libraries are bundled under `third_party/` (ANTLR4, Snappy, LZ4, LZOKAY, Zlib, SIMDe)
+- `cwrapper/` provides the C API that the Python module binds to via Cython
+
+## Code Style
+
+- **Formatter**: clang-format (Google style), configured in `.clang-format`
+
+## Testing
+
+- **Framework**: Google Test 1.12.1 (auto-downloaded during build, or supply `third_party/googletest-release-1.12.1.zip`)
+- Tests in `cpp/test/`, mirroring `src/` structure
+- Test discovery via `gtest_discover_tests()`
+
+## License Header
+
+Every new file must include the Apache License 2.0 header at the top. For C/C++ files, use the `/* */` block comment style. See any existing `.h` or `.cc` file for the exact wording.
+
+## Git Commit
+
+- Do NOT add `Co-Authored-By` trailer to commit messages.
diff --git a/java/CLAUDE.md b/java/CLAUDE.md
new file mode 100644
index 00000000..243d9e10
--- /dev/null
+++ b/java/CLAUDE.md
@@ -0,0 +1,101 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# CLAUDE.md — Java Module
+
+This file provides guidance to Claude Code (claude.ai/code) when working with the Java module.
+
+## Build Commands
+
+Maven with wrapper (`./mvnw` from repo root). All commands require `-P with-java`.
+
+```bash
+# Build (skip tests)
+./mvnw clean package -P with-java -DskipTests
+
+# Install to local Maven repo
+./mvnw install -P with-java -DskipTests
+
+# Run all tests (unit + integration)
+./mvnw clean verify -P with-java
+
+# Run unit tests only
+./mvnw test -P with-java
+
+# Run integration tests only
+./mvnw verify -P with-java -DskipUTs=true
+
+# Run a single test class (use -pl to target submodule)
+./mvnw test -P with-java -pl java/tsfile -Dtest=TsFileWriterTest
+
+# Run a single test method
+./mvnw test -P with-java -pl java/tsfile -Dtest=TsFileWriterTest#testWrite
+
+# Format code (requires Java 17+ runtime)
+./mvnw spotless:apply
+
+# Check formatting
+./mvnw spotless:check
+```
+
+## Module Structure
+
+Four submodules under `java/`:
+
+- **common** — Shared types: `TSDataType`, `ColumnCategory`, `Binary`, `BitMap`, constants
+- **tsfile** — Core implementation: read/write APIs, encoding, compression, file format
+- **examples** — Reference implementations for read/write operations
+- **tools** — CLI utilities for TsFile inspection
+
+## Architecture
+
+Key packages in `java/tsfile/src/main/java/org/apache/tsfile/`:
+
+- `write/` — Write path: `TsFileWriter`, `Tablet` (batch API), `TSRecord` (row API)
+- `read/` — Read path: `TsFileReader`, `TsFileSequenceReader`, `QueryExpression`
+- `encoding/` — Encoding algorithms (RLE, TS_2DIFF, GORILLA, DICTIONARY)
+- `compress/` — Compression codecs (Snappy, LZ4, ZSTD, XZ)
+- `file/` — File format structures (metadata, headers, footers)
+- `compatibility/` — Version compatibility handling
+- `parser/` — ANTLR4-generated path parser (grammar in `src/main/antlr4/`)
+
+Code generation: FreeMarker templates in `tsfile/src/main/codegen/` generate type-specific implementations (e.g., per-datatype readers/writers) into `target/generated-sources/`.
+
+## Code Style
+
+- **Formatter**: Spotless with Google Java Format 1.28.0 (Spotless requires Java 17+ to run but the project targets Java 8 bytecode)
+- **Checkstyle**: Google style variant in root `checkstyle.xml`, 100 char line limit
+- **Import order**: `org.apache.tsfile`, `javax`, `java`, static imports
+
+## Testing Conventions
+
+- **Framework**: JUnit 4
+- Unit tests: `*Test.java` — run via `mvn test`
+- Integration tests: `*IT.java` — run via `mvn verify`
+- Tests run in random order with `reuseForks=false`
+
+## License Header
+
+Every new file must include the Apache License 2.0 header at the top. For Java files, use the `/* */` block comment style. See any existing `.java` file for the exact wording.
+
+## Git Commit
+
+- Do NOT add `Co-Authored-By` trailer to commit messages.
diff --git a/python/CLAUDE.md b/python/CLAUDE.md
new file mode 100644
index 00000000..2fc0ca88
--- /dev/null
+++ b/python/CLAUDE.md
@@ -0,0 +1,104 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# CLAUDE.md — Python Module
+
+This file provides guidance to Claude Code (claude.ai/code) when working with the Python module.
+
+## Prerequisites
+
+The Python module depends on the C++ shared library (`libtsfile`). **Build C++ first:**
+
+```bash
+# From repo root
+./mvnw clean package -P with-cpp -DskipTests
+```
+
+## Build Commands
+
+```bash
+# Build everything via Maven (builds C++ then Python)
+./mvnw clean verify -P with-python
+
+# Or build directly (after C++ is built)
+cd python
+python setup.py build_ext --inplace
+
+# Run tests
+pytest tests/
+
+# Run a single test
+pytest tests/test_write_and_read.py -k "test_name"
+
+# Format code
+cd ../ && ./mvnw spotless:apply # Uses Black 26.3.1
+```
+
+## Source Structure
+
+```
+python/tsfile/
+├── __init__.py # Public API exports
+├── constants.py # Enums: TSDataType, TSEncoding, Compressor, ColumnCategory
+├── schema.py # TimeseriesSchema, DeviceSchema, TableSchema, ColumnSchema
+├── tablet.py # Tablet class for batch columnar data
+├── field.py / row_record.py # Row-based data structures
+├── tag_filter.py # Tag filtering DSL for queries
+├── tsfile_table_writer.py # High-level table writer API
+├── utils.py # to_dataframe(), dataframe_to_tsfile()
+├── exceptions.py # Custom exception types
+│
+├── tsfile_py_cpp.pyx # Cython core: Python ↔ C++ type conversion
+├── tsfile_reader.pyx # Cython reader wrapper
+├── tsfile_writer.pyx # Cython writer wrapper
+├── tsfile_cpp.pxd # C++ interface declarations (cwrapper)
+│
+└── dataset/ # High-level multi-file API
+ ├── dataframe.py # TsFileDataFrame (pandas-like interface)
+ ├── timeseries.py # Timeseries, AlignedTimeseries
+ ├── reader.py # Multi-file reader coordination
+ └── merge.py # K-way merge for overlapping time ranges
+```
+
+## Architecture Notes
+
+- **Layered design**: C++ core → Cython bridge (`*.pyx`) → Pure Python API
+- `setup.py` copies pre-built C++ headers and shared libraries from `../cpp/target/build/`
+- Cython binds to `cwrapper/tsfile_cwrapper.h` from the C++ module
+- C error codes are mapped to Python exceptions via `check_error()` in `tsfile_py_cpp.pyx`
+- Requires Python 3.9+, numpy >= 2.0, pandas >= 2.0, pyarrow >= 16.0
+
+## Code Style
+
+- **Formatter**: Black 26.3.1 (via Spotless Maven plugin)
+
+## Testing
+
+- **Framework**: pytest
+- Tests in `python/tests/`, 19 test files covering write, read, filtering, DataFrame conversion, and Arrow format
+
+## License Header
+
+Every new file must include the Apache License 2.0 header at the top. For Python files, use `#` line comments. For `.pyx`/`.pxd` files, also use `#` comments. See any existing file for the exact wording.
+
+## Git Commit
+
+- Do NOT add `Co-Authored-By` trailer to commit messages.