You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@iotdb.apache.org by jincheng sun <su...@gmail.com> on 2020/01/09 07:59:34 UTC

[DISCUSS] Add the components definition in JIRA template

Hi folks,

Currently we can not select one components when create a JIRA. [1]. At
present, this problem is not obvious because there are not many
contributors of IoTDB, and the active committers need to have a certain
understanding of each module. When our community grows stronger, each
module may have some specific person in charge. At that time, it is
meaningful for us to specify specific components for JIRA., because that
way, the component owner  can pay attention to them and discuss with the
contributors.

So my proposal is:
Add the components definition in JIRA template [2](there is permission
control, maybe PPMC should be able to access).

If my proposal is accepted by IoTDB community, we should have a discussion
about how to define the components for IoTDB. I'm not familiar with the
components of IoTDB. Maybe PPMC such as Xiangdong and Jialin etc.,  and the
committers of IoTDB will have better solutions.

My two cents:

Approach 1 (align the modules):
    - Client
    - Server
    - Tsfile
    - Spark-tsfile
    - Hadoop-tsfile
    - Spark-iotdb-connector
    - Hive-connector
    - Grafana
    - Distribution
    - Service-rpc
    - Jdbc
    - Session
    - Example
    - WebSite

Approach 2(Grading management):
    - Client
         - Client-java
         - Client-py
         - Service-rpc
         - Jdbc
         - Session
    - Server
    - Tsfile
          - Spark-tsfile
          - Hadoop-tsfile
    - Connector
          - Spark-iotdb-connector
          - Hive-connector
    - Grafana
    - Distribution
    - Example
    - WebSite

Approach 3(Pretty simple. We can expand it at any time):
    - Client
    - Server
    - Tsfile
    - Connector
    - Grafana
    - Distribution
    - Example
    - WebSite

Approach 4（Other better options)

Best,
Jincheng

[1] https://issues.apache.org/jira/browse/IOTDB-410
[2]
https://issues.apache.org/jira/plugins/servlet/project-config/IOTDB/components

Re: [DISCUSS] Add the components definition in JIRA template

Posted by Jialin Qiao <qi...@apache.org>.

Hi,

Attaching a component of each issue will help contributors quickly locate
the problem.
This has nothing to do with specifying someone in a certain area, it just a
classification of issues :)

I would like to give an approach 4

  - TsFile (module: tsfile)
  - Storage (StorageEngine in server module, including buffer management,
flush management)
  - Query (Query processor in server module)
  - Merge (Compaction in server)
  - CLI (module: iotdb-client)
  - Rpc-client (modules: iotdb-jdbc, service-rpc, iotdb-session)
  - Python-client
  - Distribution
  - Grafana
  - Spark-TsFile
  - Spark-IoTDB
  - Hive-tsfile
  - Hadoop-tsfile
  - Example
  - WebSite
  - Doc
  - Other

Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 下午4:27写道：

> Hi,
>
> > When our community grows stronger, each\ module may have some specific
> person in charge.
>
> No that is not the Apache Way. No one should be in charge of an area.
>
> Thanks,
> Justin



-- 
—————————————————
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

Re: [DISCUSS] Add the components definition in JIRA template

Posted by Jialin Qiao <qj...@mails.tsinghua.edu.cn>.

Hi,

+1 for a vote

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -----原始邮件-----
> 发件人: "jincheng sun" <su...@gmail.com>
> 发送时间: 2020-01-24 12:32:22 (星期五)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: [DISCUSS] Add the components definition in JIRA template
> 
> I would like to bring up a VOTE mail to make the final decision If there no
> objections.
> 
> What do you think?
> 
> Best,
> Jincheng
> -------------
> Twitter: https://twitter.com/sunjincheng121
> -------------
> 
> 
> jincheng sun <su...@gmail.com> 于2020年1月17日周五 上午10:39写道：
> 
> > Great thanks for adding more relationship info about `Merge`  and
> > `Storage`.  @Xiangdong !
> >
> > As :
> > > (now query engine is in the storage engine..) Merge is a submodule of
> > storage engine and query engine.
> >
> > I think `query engine` and `storage engine` is make sense to me. Here only
> > a suggestion that how about use the `Engine` for both `Storage Engine` and
> > `Query Engine` as they are very close each other and they also have the
> > common share `Merge`. i.e.,:
> >
> > --------
> > Core
> > Core/TsFile
> > Core/Engine
> > Core/WAL
> > ---------------
> >
> > What to do you think?
> >
> > Best,
> > Jincheng
> >
> >
> > ------------------------------------------------------
> > 1. --Connectors-------------
> > Connectors components include all data integration with external systems,
> > The components are listed as follows:
> >
> > Connectors
> > Connectors/Grafana
> > Connectors/Spark-TsFile
> > Connectors/Spark-IoTDB
> > Connectors/Hive-tsfile
> > Connectors/Hadoop-tsfile
> >
> > 2. --Client-------------
> > Client components involves RPC between client and server, a command-line
> > interface（multi-languages).The components are listed as follows:
> >
> > Client
> > Client/Java
> > Client/Python
> >
> > 3. --Planner-------------
> > Planner components involves the query process in IoTDB, including multiple
> > types of queries, such as raw data query, aggregations, group by time,
> > group by device, and includes many optimized rules.The components are
> > listed as follows:
> >
> > Planner
> > Planner/sql parser
> > Planner/sql optimizer
> >
> > 4. --Core-------------
> > Core components is the storage engine of IoTDB. The components are listed
> > as follows:
> >
> > Core
> > Core/TsFile
> > Core/Engine
> > Core/WAL
> >
> > 5. --Tools-------------
> > Tools components involves many utils of IoTDB, Such as data Sync. The
> > components are listed as follows:
> > Tools
> > Tools/Sync
> > Tools/Other
> >
> > 6. --Authority---------
> > Authority
> >
> > 7. --Distribution------
> > Distribution
> >
> > 8. --Example-----------
> > Example
> >
> > 9. --WebSite-----------
> > WebSite
> >
> > 10. --Doc---------------
> > Doc
> >
> > 11. --Other------------
> > Other （I'm not sure if we really need the Other component, but there is no
> > harm in adding one）
> >
> > Xiangdong Huang <sa...@gmail.com> 于2020年1月17日周五 上午12:50写道：
> >
> >> Hi,
> >>
> >> Thanks jincheng and jialin.
> >>
> >> Sorry for absence of the discussion because of  busy work these days..
> >>
> >> Some comments:
> >>
> >> > Core
> >> > Core/TsFile
> >> > Core/Merge
> >> > Core/Storage
> >> > Core/WAL
> >>
> >> Not so clearly.. maybe TsFile, WAL, Storage Engine, Query Engine? (now
> >> query engine is in the storage engine..)
> >> Merge is a submodule of storage engine and query engine.
> >> If we pick out the merge module, we also need to pick out the FLUSH,
> >> MODIFICATION, etc.. I think.
> >>
> >> Best,
> >> -----------------------------------
> >> Xiangdong Huang
> >> School of Software, Tsinghua University
> >>
> >>  黄向东
> >> 清华大学 软件学院
> >>
> >>
> >> Jialin Qiao <qi...@apache.org> 于2020年1月16日周四 下午11:21写道：
> >>
> >> > Hi Jincheng,
> >> >
> >> > Thank you! Looks good :)
> >> >
> >> > jincheng sun <su...@gmail.com> 于2020年1月16日周四 下午4:01写道：
> >> >
> >> > > Thanks for your explanation! @Jialin
> >> > >
> >> > > I thought about the composition of our components from the external
> >> > system,
> >> > > the user level, to the IoTDB bottom-level implementation to the
> >> auxiliary
> >> > > tools, and I adjusted a little based on yours.  as follows:
> >> > >
> >> > > 1. --Connectors-------------
> >> > > Connectors components include all data integration with external
> >> systems,
> >> > > The components are listed as follows:
> >> > >
> >> > > Connectors
> >> > > Connectors/Grafana
> >> > > Connectors/Spark-TsFile
> >> > > Connectors/Spark-IoTDB
> >> > > Connectors/Hive-tsfile
> >> > > Connectors/Hadoop-tsfile
> >> > >
> >> > > 2. --Client-------------
> >> > > Client components involves RPC between client and server, a
> >> command-line
> >> > > interface（multi-languages).The components are listed as follows:
> >> > >
> >> > > Client
> >> > > Client/Java
> >> > > Client/Python
> >> > >
> >> > > 3. --Planner-------------
> >> > > Planner components involves the query process in IoTDB, including
> >> > multiple
> >> > > types of queries, such as raw data query, aggregations, group by time,
> >> > > group by device, and includes many optimized rules.The components are
> >> > > listed as follows:
> >> > >
> >> > > Planner
> >> > > Planner/sql parsar
> >> > > Planner/sql optimizer
> >> > >
> >> > > 4. --Core-------------
> >> > > Core components is the storage engine of IoTDB. The components are
> >> listed
> >> > > as follows:
> >> > >
> >> > > Core
> >> > > Core/TsFile
> >> > > Core/Merge
> >> > > Core/Storage
> >> > > Core/WAL
> >> > >
> >> > > 5. --Tools-------------
> >> > > Tools components involves many utils of IoTDB, Such as data Sync. The
> >> > > components are listed as follows:
> >> > > Tools
> >> > > Tools/Sync
> >> > > Tools/Other
> >> > >
> >> > > 6. --Authority---------
> >> > > Authority
> >> > >
> >> > > 7. --Distribution------
> >> > > Distribution
> >> > >
> >> > > 8. --Example-----------
> >> > > Example
> >> > >
> >> > > 9. --WebSite-----------
> >> > > WebSite
> >> > >
> >> > > 10. --Doc---------------
> >> > > Doc
> >> > >
> >> > > 11. --Other------------
> >> > > Other （I'm not sure if we really need the Other component, but there
> >> is
> >> > no
> >> > > harm in adding one）
> >> > >
> >> > > This does not change much from your solution. The core is to add a
> >> layer
> >> > of
> >> > > abstraction.
> >> > >
> >> > > Does this make sense to you? Anyway we can continue to improve if
> >> needed!
> >> > >
> >> > > Best,
> >> > > Jincheng
> >> > >
> >> > > Jialin Qiao <qi...@apache.org> 于2020年1月15日周三 下午4:28写道：
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > WAL has an independent physical layout that has nothing to do with
> >> > > TsFile.
> >> > > > It is used in the writing process and recovering in IoTDB.
> >> > > > It could be merged into Storage components.
> >> > > >
> >> > > > I'm not sure whether I leave out some components, so I add an Other
> >> and
> >> > > > therefore can not give an example...
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > jincheng sun <su...@gmail.com> 于2020年1月15日周三 下午4:01写道：
> >> > > >
> >> > > > > Sorry for the late reply, I just came back from team outing.
> >> > > > >
> >> > > > > Thanks for adding more detail information. I have two questions
> >> then:
> >> > > > >
> >> > > > > - Is WAL a write strategy for TsFlie, can WAL be incorporated into
> >> > > other
> >> > > > > components and what is the reason for being a separate component?
> >> > > > > - Can you give an example of what changes belongs to the Other
> >> > > component?
> >> > > > >
> >> > > > > Best,
> >> > > > > Jincheng
> >> > > > >
> >> > > > >
> >> > > > > Jialin Qiao <qi...@apache.org> 于2020年1月9日周四 下午6:39写道：
> >> > > > >
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > Sure, maybe a bit more...
> >> > > > > >
> >> > > > > >   - TsFile : A columnar file format that has flexible schema and
> >> > data
> >> > > > > > management for time series.
> >> > > > > >   - Storage : The storage engine of IoTDB, including writing to
> >> > > > memtable
> >> > > > > > (buffer), flushing, system recovery, and memtable management.
> >> > > > > >   - Query : The query process in IoTDB, including multiple
> >> types of
> >> > > > > > queries, such as raw data query, aggregations, group by time,
> >> group
> >> > > by
> >> > > > > > device.
> >> > > > > >   - Merge : Compaction of data files, including multiple
> >> compaction
> >> > > > > > strategies, such as (1) merging unsequence TsFile with sequence
> >> > > TsFile,
> >> > > > > (2)
> >> > > > > > merging small sequence TsFiles into a large TsFile.
> >> > > > > >   - Sync: Synchronize data files from one IoTDB instance to
> >> > another.
> >> > > > > >   - Authority: Administration of IoTDB, including users, roles
> >> and
> >> > > > > > privileges.
> >> > > > > >   - WAL : Write-ahead-log.
> >> > > > > >   - Client: Rpc between client and server, a command-line
> >> interface
> >> > > and
> >> > > > > > client of different languages.
> >> > > > > >   - Distribution : The shared-nothing distribution of IoTDB,
> >> > > including
> >> > > > > raft
> >> > > > > > protocol, data partition, and system information management.
> >> > > > > >   - Grafana : The connector for using IoTDB as a data source in
> >> > > > Grafana.
> >> > > > > >   - Spark-TsFile : A connector that allows users read/write
> >> TsFile
> >> > > > > through
> >> > > > > > SparkSQL (treat TsFile as an external data source for SparkSQL).
> >> > > > > >   - Spark-IoTDB : A connector that allows users read/write IoTDB
> >> > > > through
> >> > > > > > SparkSQL (treat IoTDB as an external data source for SparkSQL).
> >> > > > > >   - Hive-tsfile : A connector that maps TsFile as a table in
> >> Hive
> >> > for
> >> > > > the
> >> > > > > > query.
> >> > > > > >   - Hadoop-tsfile : A connector that allows users read/write
> >> TsFile
> >> > > > > through
> >> > > > > > MapReduce.
> >> > > > > >   - Tools : Other tools for IoTDB, such as watermark, memory
> >> > > > estimation,
> >> > > > > > JMX, query history visualization.
> >> > > > > >   - Example : Examples of different modules, such as JDBC,
> >> Session,
> >> > > > > TsFile.
> >> > > > > >   - WebSite : IoTDB official website.
> >> > > > > >   - Doc: documents of IoTDB, docs folder under root directory.
> >> > > > > >   - Other: Those do not belong to the above components.
> >> > > > > >
> >> > > > > >
> >> > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午5:28写道：
> >> > > > > >
> >> > > > > > > Hi,
> >> > > > > > >
> >> > > > > > > @Xiangdong thank you for helping to clarify my original
> >> > intention.
> >> > > > you
> >> > > > > > are
> >> > > > > > > right, I really want to list some components to let users
> >> choose
> >> > > > which
> >> > > > > > > component the given issue belongs to.
> >> > > > > > >
> >> > > > > > > @Jialin I think approach 4 would be more meaningful for IoTDB.
> >> > > Could
> >> > > > > you
> >> > > > > > > please add the detailed description for each component, such
> >> as,
> >> > > > > describe
> >> > > > > > > the relationship between `tsfile` and
> >> > > > > > > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which
> >> > > changes
> >> > > > > > belong
> >> > > > > > > to `Other` component, etc., which can help us better
> >> understand
> >> > the
> >> > > > > > reasons
> >> > > > > > > for such classification. On the other hand, we will further
> >> > discuss
> >> > > > > > whether
> >> > > > > > > there is optimization space for this component classification.
> >> > > > > > >
> >> > > > > > > What do you think?
> >> > > > > > >
> >> > > > > > > Best,
> >> > > > > > > Jincheng
> >> > > > > > >
> >> > > > > > > Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
> >> > > > > > >
> >> > > > > > > > >  No that is not the Apache Way. No one should be in
> >> charge of
> >> > > an
> >> > > > > area
> >> > > > > > > >
> >> > > > > > > > I think Jincheng's mean is, we can list some components to
> >> let
> >> > > > users
> >> > > > > > > > choose which component the given issue belongs to.
> >> > > > > > > >
> >> > > > > > > > As for which components should be listed, we need to have a
> >> > > > > discussion.
> >> > > > > > > >
> >> > > > > > > > > No one should be in charge of an area
> >> > > > > > > >
> >> > > > > > > > Yes everyone can assign an issue to himself/herself.
> >> > > > > > > >
> >> > > > > > > > Best,
> >> > > > > > > > ----------------------------------
> >> > > > > > > > Xiangdong Huang
> >> > > > > > > > School of Software, Tsinghua University
> >> > > > > > > >
> >> > > > > > > >  黄向东
> >> > > > > > > > 清华大学 软件学院
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四
> >> 下午4:38写道：
> >> > > > > > > >
> >> > > > > > > > > > No that is not the Apache Way. No one should be in
> >> charge
> >> > of
> >> > > an
> >> > > > > > area.
> >> > > > > > > > >
> >> > > > > > > > > Oh, great thanks for correcting me Justin, maybe it's my
> >> > > language
> >> > > > > > > > > expression problems. I mean some contributors and
> >> committers
> >> > > will
> >> > > > > be
> >> > > > > > > very
> >> > > > > > > > > interested in some components, then spend more time, and
> >> be
> >> > > more
> >> > > > > > > familiar
> >> > > > > > > > > with the code.
> >> > > > > > > > >
> >> > > > > > > > > Best,
> >> > > > > > > > > Jincheng
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四
> >> > > 下午4:27写道：
> >> > > > > > > > >
> >> > > > > > > > > > Hi,
> >> > > > > > > > > >
> >> > > > > > > > > > > When our community grows stronger, each\ module may
> >> have
> >> > > some
> >> > > > > > > > specific
> >> > > > > > > > > > person in charge.
> >> > > > > > > > > >
> >> > > > > > > > > > No that is not the Apache Way. No one should be in
> >> charge
> >> > of
> >> > > an
> >> > > > > > area.
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks,
> >> > > > > > > > > > Justin
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > > —————————————————
> >> > > > > > Jialin Qiao
> >> > > > > > School of Software, Tsinghua University
> >> > > > > >
> >> > > > > > 乔嘉林
> >> > > > > > 清华大学 软件学院
> >> > > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > —————————————————
> >> > > > Jialin Qiao
> >> > > > School of Software, Tsinghua University
> >> > > >
> >> > > > 乔嘉林
> >> > > > 清华大学 软件学院
> >> > > >
> >> > >
> >> >
> >> >
> >> > --
> >> > —————————————————
> >> > Jialin Qiao
> >> > School of Software, Tsinghua University
> >> >
> >> > 乔嘉林
> >> > 清华大学 软件学院
> >> >
> >>
> >

Re: [DISCUSS] Add the components definition in JIRA template

Posted by Xiangdong Huang <sa...@gmail.com>.

+ 1

jincheng sun <su...@gmail.com>于2020年1月24日 周五下午12:33写道：

> I would like to bring up a VOTE mail to make the final decision If there no
> objections.
>
> What do you think?
>
> Best,
> Jincheng
> -------------
> Twitter: https://twitter.com/sunjincheng121
> -------------
>
>
> jincheng sun <su...@gmail.com> 于2020年1月17日周五 上午10:39写道：
>
> > Great thanks for adding more relationship info about `Merge`  and
> > `Storage`.  @Xiangdong !
> >
> > As :
> > > (now query engine is in the storage engine..) Merge is a submodule of
> > storage engine and query engine.
> >
> > I think `query engine` and `storage engine` is make sense to me. Here
> only
> > a suggestion that how about use the `Engine` for both `Storage Engine`
> and
> > `Query Engine` as they are very close each other and they also have the
> > common share `Merge`. i.e.,:
> >
> > --------
> > Core
> > Core/TsFile
> > Core/Engine
> > Core/WAL
> > ---------------
> >
> > What to do you think?
> >
> > Best,
> > Jincheng
> >
> >
> > ------------------------------------------------------
> > 1. --Connectors-------------
> > Connectors components include all data integration with external systems,
> > The components are listed as follows:
> >
> > Connectors
> > Connectors/Grafana
> > Connectors/Spark-TsFile
> > Connectors/Spark-IoTDB
> > Connectors/Hive-tsfile
> > Connectors/Hadoop-tsfile
> >
> > 2. --Client-------------
> > Client components involves RPC between client and server, a command-line
> > interface（multi-languages).The components are listed as follows:
> >
> > Client
> > Client/Java
> > Client/Python
> >
> > 3. --Planner-------------
> > Planner components involves the query process in IoTDB, including
> multiple
> > types of queries, such as raw data query, aggregations, group by time,
> > group by device, and includes many optimized rules.The components are
> > listed as follows:
> >
> > Planner
> > Planner/sql parser
> > Planner/sql optimizer
> >
> > 4. --Core-------------
> > Core components is the storage engine of IoTDB. The components are listed
> > as follows:
> >
> > Core
> > Core/TsFile
> > Core/Engine
> > Core/WAL
> >
> > 5. --Tools-------------
> > Tools components involves many utils of IoTDB, Such as data Sync. The
> > components are listed as follows:
> > Tools
> > Tools/Sync
> > Tools/Other
> >
> > 6. --Authority---------
> > Authority
> >
> > 7. --Distribution------
> > Distribution
> >
> > 8. --Example-----------
> > Example
> >
> > 9. --WebSite-----------
> > WebSite
> >
> > 10. --Doc---------------
> > Doc
> >
> > 11. --Other------------
> > Other （I'm not sure if we really need the Other component, but there is
> no
> > harm in adding one）
> >
> > Xiangdong Huang <sa...@gmail.com> 于2020年1月17日周五 上午12:50写道：
> >
> >> Hi,
> >>
> >> Thanks jincheng and jialin.
> >>
> >> Sorry for absence of the discussion because of  busy work these days..
> >>
> >> Some comments:
> >>
> >> > Core
> >> > Core/TsFile
> >> > Core/Merge
> >> > Core/Storage
> >> > Core/WAL
> >>
> >> Not so clearly.. maybe TsFile, WAL, Storage Engine, Query Engine? (now
> >> query engine is in the storage engine..)
> >> Merge is a submodule of storage engine and query engine.
> >> If we pick out the merge module, we also need to pick out the FLUSH,
> >> MODIFICATION, etc.. I think.
> >>
> >> Best,
> >> -----------------------------------
> >> Xiangdong Huang
> >> School of Software, Tsinghua University
> >>
> >>  黄向东
> >> 清华大学 软件学院
> >>
> >>
> >> Jialin Qiao <qi...@apache.org> 于2020年1月16日周四 下午11:21写道：
> >>
> >> > Hi Jincheng,
> >> >
> >> > Thank you! Looks good :)
> >> >
> >> > jincheng sun <su...@gmail.com> 于2020年1月16日周四 下午4:01写道：
> >> >
> >> > > Thanks for your explanation! @Jialin
> >> > >
> >> > > I thought about the composition of our components from the external
> >> > system,
> >> > > the user level, to the IoTDB bottom-level implementation to the
> >> auxiliary
> >> > > tools, and I adjusted a little based on yours.  as follows:
> >> > >
> >> > > 1. --Connectors-------------
> >> > > Connectors components include all data integration with external
> >> systems,
> >> > > The components are listed as follows:
> >> > >
> >> > > Connectors
> >> > > Connectors/Grafana
> >> > > Connectors/Spark-TsFile
> >> > > Connectors/Spark-IoTDB
> >> > > Connectors/Hive-tsfile
> >> > > Connectors/Hadoop-tsfile
> >> > >
> >> > > 2. --Client-------------
> >> > > Client components involves RPC between client and server, a
> >> command-line
> >> > > interface（multi-languages).The components are listed as follows:
> >> > >
> >> > > Client
> >> > > Client/Java
> >> > > Client/Python
> >> > >
> >> > > 3. --Planner-------------
> >> > > Planner components involves the query process in IoTDB, including
> >> > multiple
> >> > > types of queries, such as raw data query, aggregations, group by
> time,
> >> > > group by device, and includes many optimized rules.The components
> are
> >> > > listed as follows:
> >> > >
> >> > > Planner
> >> > > Planner/sql parsar
> >> > > Planner/sql optimizer
> >> > >
> >> > > 4. --Core-------------
> >> > > Core components is the storage engine of IoTDB. The components are
> >> listed
> >> > > as follows:
> >> > >
> >> > > Core
> >> > > Core/TsFile
> >> > > Core/Merge
> >> > > Core/Storage
> >> > > Core/WAL
> >> > >
> >> > > 5. --Tools-------------
> >> > > Tools components involves many utils of IoTDB, Such as data Sync.
> The
> >> > > components are listed as follows:
> >> > > Tools
> >> > > Tools/Sync
> >> > > Tools/Other
> >> > >
> >> > > 6. --Authority---------
> >> > > Authority
> >> > >
> >> > > 7. --Distribution------
> >> > > Distribution
> >> > >
> >> > > 8. --Example-----------
> >> > > Example
> >> > >
> >> > > 9. --WebSite-----------
> >> > > WebSite
> >> > >
> >> > > 10. --Doc---------------
> >> > > Doc
> >> > >
> >> > > 11. --Other------------
> >> > > Other （I'm not sure if we really need the Other component, but there
> >> is
> >> > no
> >> > > harm in adding one）
> >> > >
> >> > > This does not change much from your solution. The core is to add a
> >> layer
> >> > of
> >> > > abstraction.
> >> > >
> >> > > Does this make sense to you? Anyway we can continue to improve if
> >> needed!
> >> > >
> >> > > Best,
> >> > > Jincheng
> >> > >
> >> > > Jialin Qiao <qi...@apache.org> 于2020年1月15日周三 下午4:28写道：
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > WAL has an independent physical layout that has nothing to do with
> >> > > TsFile.
> >> > > > It is used in the writing process and recovering in IoTDB.
> >> > > > It could be merged into Storage components.
> >> > > >
> >> > > > I'm not sure whether I leave out some components, so I add an
> Other
> >> and
> >> > > > therefore can not give an example...
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > jincheng sun <su...@gmail.com> 于2020年1月15日周三 下午4:01写道：
> >> > > >
> >> > > > > Sorry for the late reply, I just came back from team outing.
> >> > > > >
> >> > > > > Thanks for adding more detail information. I have two questions
> >> then:
> >> > > > >
> >> > > > > - Is WAL a write strategy for TsFlie, can WAL be incorporated
> into
> >> > > other
> >> > > > > components and what is the reason for being a separate
> component?
> >> > > > > - Can you give an example of what changes belongs to the Other
> >> > > component?
> >> > > > >
> >> > > > > Best,
> >> > > > > Jincheng
> >> > > > >
> >> > > > >
> >> > > > > Jialin Qiao <qi...@apache.org> 于2020年1月9日周四 下午6:39写道：
> >> > > > >
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > Sure, maybe a bit more...
> >> > > > > >
> >> > > > > >   - TsFile : A columnar file format that has flexible schema
> and
> >> > data
> >> > > > > > management for time series.
> >> > > > > >   - Storage : The storage engine of IoTDB, including writing
> to
> >> > > > memtable
> >> > > > > > (buffer), flushing, system recovery, and memtable management.
> >> > > > > >   - Query : The query process in IoTDB, including multiple
> >> types of
> >> > > > > > queries, such as raw data query, aggregations, group by time,
> >> group
> >> > > by
> >> > > > > > device.
> >> > > > > >   - Merge : Compaction of data files, including multiple
> >> compaction
> >> > > > > > strategies, such as (1) merging unsequence TsFile with
> sequence
> >> > > TsFile,
> >> > > > > (2)
> >> > > > > > merging small sequence TsFiles into a large TsFile.
> >> > > > > >   - Sync: Synchronize data files from one IoTDB instance to
> >> > another.
> >> > > > > >   - Authority: Administration of IoTDB, including users, roles
> >> and
> >> > > > > > privileges.
> >> > > > > >   - WAL : Write-ahead-log.
> >> > > > > >   - Client: Rpc between client and server, a command-line
> >> interface
> >> > > and
> >> > > > > > client of different languages.
> >> > > > > >   - Distribution : The shared-nothing distribution of IoTDB,
> >> > > including
> >> > > > > raft
> >> > > > > > protocol, data partition, and system information management.
> >> > > > > >   - Grafana : The connector for using IoTDB as a data source
> in
> >> > > > Grafana.
> >> > > > > >   - Spark-TsFile : A connector that allows users read/write
> >> TsFile
> >> > > > > through
> >> > > > > > SparkSQL (treat TsFile as an external data source for
> SparkSQL).
> >> > > > > >   - Spark-IoTDB : A connector that allows users read/write
> IoTDB
> >> > > > through
> >> > > > > > SparkSQL (treat IoTDB as an external data source for
> SparkSQL).
> >> > > > > >   - Hive-tsfile : A connector that maps TsFile as a table in
> >> Hive
> >> > for
> >> > > > the
> >> > > > > > query.
> >> > > > > >   - Hadoop-tsfile : A connector that allows users read/write
> >> TsFile
> >> > > > > through
> >> > > > > > MapReduce.
> >> > > > > >   - Tools : Other tools for IoTDB, such as watermark, memory
> >> > > > estimation,
> >> > > > > > JMX, query history visualization.
> >> > > > > >   - Example : Examples of different modules, such as JDBC,
> >> Session,
> >> > > > > TsFile.
> >> > > > > >   - WebSite : IoTDB official website.
> >> > > > > >   - Doc: documents of IoTDB, docs folder under root directory.
> >> > > > > >   - Other: Those do not belong to the above components.
> >> > > > > >
> >> > > > > >
> >> > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四
> 下午5:28写道：
> >> > > > > >
> >> > > > > > > Hi,
> >> > > > > > >
> >> > > > > > > @Xiangdong thank you for helping to clarify my original
> >> > intention.
> >> > > > you
> >> > > > > > are
> >> > > > > > > right, I really want to list some components to let users
> >> choose
> >> > > > which
> >> > > > > > > component the given issue belongs to.
> >> > > > > > >
> >> > > > > > > @Jialin I think approach 4 would be more meaningful for
> IoTDB.
> >> > > Could
> >> > > > > you
> >> > > > > > > please add the detailed description for each component, such
> >> as,
> >> > > > > describe
> >> > > > > > > the relationship between `tsfile` and
> >> > > > > > > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which
> >> > > changes
> >> > > > > > belong
> >> > > > > > > to `Other` component, etc., which can help us better
> >> understand
> >> > the
> >> > > > > > reasons
> >> > > > > > > for such classification. On the other hand, we will further
> >> > discuss
> >> > > > > > whether
> >> > > > > > > there is optimization space for this component
> classification.
> >> > > > > > >
> >> > > > > > > What do you think?
> >> > > > > > >
> >> > > > > > > Best,
> >> > > > > > > Jincheng
> >> > > > > > >
> >> > > > > > > Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
> >> > > > > > >
> >> > > > > > > > >  No that is not the Apache Way. No one should be in
> >> charge of
> >> > > an
> >> > > > > area
> >> > > > > > > >
> >> > > > > > > > I think Jincheng's mean is, we can list some components to
> >> let
> >> > > > users
> >> > > > > > > > choose which component the given issue belongs to.
> >> > > > > > > >
> >> > > > > > > > As for which components should be listed, we need to have
> a
> >> > > > > discussion.
> >> > > > > > > >
> >> > > > > > > > > No one should be in charge of an area
> >> > > > > > > >
> >> > > > > > > > Yes everyone can assign an issue to himself/herself.
> >> > > > > > > >
> >> > > > > > > > Best,
> >> > > > > > > > ----------------------------------
> >> > > > > > > > Xiangdong Huang
> >> > > > > > > > School of Software, Tsinghua University
> >> > > > > > > >
> >> > > > > > > >  黄向东
> >> > > > > > > > 清华大学 软件学院
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四
> >> 下午4:38写道：
> >> > > > > > > >
> >> > > > > > > > > > No that is not the Apache Way. No one should be in
> >> charge
> >> > of
> >> > > an
> >> > > > > > area.
> >> > > > > > > > >
> >> > > > > > > > > Oh, great thanks for correcting me Justin, maybe it's my
> >> > > language
> >> > > > > > > > > expression problems. I mean some contributors and
> >> committers
> >> > > will
> >> > > > > be
> >> > > > > > > very
> >> > > > > > > > > interested in some components, then spend more time, and
> >> be
> >> > > more
> >> > > > > > > familiar
> >> > > > > > > > > with the code.
> >> > > > > > > > >
> >> > > > > > > > > Best,
> >> > > > > > > > > Jincheng
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四
> >> > > 下午4:27写道：
> >> > > > > > > > >
> >> > > > > > > > > > Hi,
> >> > > > > > > > > >
> >> > > > > > > > > > > When our community grows stronger, each\ module may
> >> have
> >> > > some
> >> > > > > > > > specific
> >> > > > > > > > > > person in charge.
> >> > > > > > > > > >
> >> > > > > > > > > > No that is not the Apache Way. No one should be in
> >> charge
> >> > of
> >> > > an
> >> > > > > > area.
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks,
> >> > > > > > > > > > Justin
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > > —————————————————
> >> > > > > > Jialin Qiao
> >> > > > > > School of Software, Tsinghua University
> >> > > > > >
> >> > > > > > 乔嘉林
> >> > > > > > 清华大学 软件学院
> >> > > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > —————————————————
> >> > > > Jialin Qiao
> >> > > > School of Software, Tsinghua University
> >> > > >
> >> > > > 乔嘉林
> >> > > > 清华大学 软件学院
> >> > > >
> >> > >
> >> >
> >> >
> >> > --
> >> > —————————————————
> >> > Jialin Qiao
> >> > School of Software, Tsinghua University
> >> >
> >> > 乔嘉林
> >> > 清华大学 软件学院
> >> >
> >>
> >
>
-- 
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院

Re: [DISCUSS] Add the components definition in JIRA template

Posted by jincheng sun <su...@gmail.com>.

I would like to bring up a VOTE mail to make the final decision If there no
objections.

What do you think?

Best,
Jincheng
-------------
Twitter: https://twitter.com/sunjincheng121
-------------


jincheng sun <su...@gmail.com> 于2020年1月17日周五 上午10:39写道：

> Great thanks for adding more relationship info about `Merge`  and
> `Storage`.  @Xiangdong !
>
> As :
> > (now query engine is in the storage engine..) Merge is a submodule of
> storage engine and query engine.
>
> I think `query engine` and `storage engine` is make sense to me. Here only
> a suggestion that how about use the `Engine` for both `Storage Engine` and
> `Query Engine` as they are very close each other and they also have the
> common share `Merge`. i.e.,:
>
> --------
> Core
> Core/TsFile
> Core/Engine
> Core/WAL
> ---------------
>
> What to do you think?
>
> Best,
> Jincheng
>
>
> ------------------------------------------------------
> 1. --Connectors-------------
> Connectors components include all data integration with external systems,
> The components are listed as follows:
>
> Connectors
> Connectors/Grafana
> Connectors/Spark-TsFile
> Connectors/Spark-IoTDB
> Connectors/Hive-tsfile
> Connectors/Hadoop-tsfile
>
> 2. --Client-------------
> Client components involves RPC between client and server, a command-line
> interface（multi-languages).The components are listed as follows:
>
> Client
> Client/Java
> Client/Python
>
> 3. --Planner-------------
> Planner components involves the query process in IoTDB, including multiple
> types of queries, such as raw data query, aggregations, group by time,
> group by device, and includes many optimized rules.The components are
> listed as follows:
>
> Planner
> Planner/sql parser
> Planner/sql optimizer
>
> 4. --Core-------------
> Core components is the storage engine of IoTDB. The components are listed
> as follows:
>
> Core
> Core/TsFile
> Core/Engine
> Core/WAL
>
> 5. --Tools-------------
> Tools components involves many utils of IoTDB, Such as data Sync. The
> components are listed as follows:
> Tools
> Tools/Sync
> Tools/Other
>
> 6. --Authority---------
> Authority
>
> 7. --Distribution------
> Distribution
>
> 8. --Example-----------
> Example
>
> 9. --WebSite-----------
> WebSite
>
> 10. --Doc---------------
> Doc
>
> 11. --Other------------
> Other （I'm not sure if we really need the Other component, but there is no
> harm in adding one）
>
> Xiangdong Huang <sa...@gmail.com> 于2020年1月17日周五 上午12:50写道：
>
>> Hi,
>>
>> Thanks jincheng and jialin.
>>
>> Sorry for absence of the discussion because of  busy work these days..
>>
>> Some comments:
>>
>> > Core
>> > Core/TsFile
>> > Core/Merge
>> > Core/Storage
>> > Core/WAL
>>
>> Not so clearly.. maybe TsFile, WAL, Storage Engine, Query Engine? (now
>> query engine is in the storage engine..)
>> Merge is a submodule of storage engine and query engine.
>> If we pick out the merge module, we also need to pick out the FLUSH,
>> MODIFICATION, etc.. I think.
>>
>> Best,
>> -----------------------------------
>> Xiangdong Huang
>> School of Software, Tsinghua University
>>
>>  黄向东
>> 清华大学 软件学院
>>
>>
>> Jialin Qiao <qi...@apache.org> 于2020年1月16日周四 下午11:21写道：
>>
>> > Hi Jincheng,
>> >
>> > Thank you! Looks good :)
>> >
>> > jincheng sun <su...@gmail.com> 于2020年1月16日周四 下午4:01写道：
>> >
>> > > Thanks for your explanation! @Jialin
>> > >
>> > > I thought about the composition of our components from the external
>> > system,
>> > > the user level, to the IoTDB bottom-level implementation to the
>> auxiliary
>> > > tools, and I adjusted a little based on yours.  as follows:
>> > >
>> > > 1. --Connectors-------------
>> > > Connectors components include all data integration with external
>> systems,
>> > > The components are listed as follows:
>> > >
>> > > Connectors
>> > > Connectors/Grafana
>> > > Connectors/Spark-TsFile
>> > > Connectors/Spark-IoTDB
>> > > Connectors/Hive-tsfile
>> > > Connectors/Hadoop-tsfile
>> > >
>> > > 2. --Client-------------
>> > > Client components involves RPC between client and server, a
>> command-line
>> > > interface（multi-languages).The components are listed as follows:
>> > >
>> > > Client
>> > > Client/Java
>> > > Client/Python
>> > >
>> > > 3. --Planner-------------
>> > > Planner components involves the query process in IoTDB, including
>> > multiple
>> > > types of queries, such as raw data query, aggregations, group by time,
>> > > group by device, and includes many optimized rules.The components are
>> > > listed as follows:
>> > >
>> > > Planner
>> > > Planner/sql parsar
>> > > Planner/sql optimizer
>> > >
>> > > 4. --Core-------------
>> > > Core components is the storage engine of IoTDB. The components are
>> listed
>> > > as follows:
>> > >
>> > > Core
>> > > Core/TsFile
>> > > Core/Merge
>> > > Core/Storage
>> > > Core/WAL
>> > >
>> > > 5. --Tools-------------
>> > > Tools components involves many utils of IoTDB, Such as data Sync. The
>> > > components are listed as follows:
>> > > Tools
>> > > Tools/Sync
>> > > Tools/Other
>> > >
>> > > 6. --Authority---------
>> > > Authority
>> > >
>> > > 7. --Distribution------
>> > > Distribution
>> > >
>> > > 8. --Example-----------
>> > > Example
>> > >
>> > > 9. --WebSite-----------
>> > > WebSite
>> > >
>> > > 10. --Doc---------------
>> > > Doc
>> > >
>> > > 11. --Other------------
>> > > Other （I'm not sure if we really need the Other component, but there
>> is
>> > no
>> > > harm in adding one）
>> > >
>> > > This does not change much from your solution. The core is to add a
>> layer
>> > of
>> > > abstraction.
>> > >
>> > > Does this make sense to you? Anyway we can continue to improve if
>> needed!
>> > >
>> > > Best,
>> > > Jincheng
>> > >
>> > > Jialin Qiao <qi...@apache.org> 于2020年1月15日周三 下午4:28写道：
>> > >
>> > > > Hi,
>> > > >
>> > > > WAL has an independent physical layout that has nothing to do with
>> > > TsFile.
>> > > > It is used in the writing process and recovering in IoTDB.
>> > > > It could be merged into Storage components.
>> > > >
>> > > > I'm not sure whether I leave out some components, so I add an Other
>> and
>> > > > therefore can not give an example...
>> > > >
>> > > > Thanks,
>> > > >
>> > > > jincheng sun <su...@gmail.com> 于2020年1月15日周三 下午4:01写道：
>> > > >
>> > > > > Sorry for the late reply, I just came back from team outing.
>> > > > >
>> > > > > Thanks for adding more detail information. I have two questions
>> then:
>> > > > >
>> > > > > - Is WAL a write strategy for TsFlie, can WAL be incorporated into
>> > > other
>> > > > > components and what is the reason for being a separate component?
>> > > > > - Can you give an example of what changes belongs to the Other
>> > > component?
>> > > > >
>> > > > > Best,
>> > > > > Jincheng
>> > > > >
>> > > > >
>> > > > > Jialin Qiao <qi...@apache.org> 于2020年1月9日周四 下午6:39写道：
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > Sure, maybe a bit more...
>> > > > > >
>> > > > > >   - TsFile : A columnar file format that has flexible schema and
>> > data
>> > > > > > management for time series.
>> > > > > >   - Storage : The storage engine of IoTDB, including writing to
>> > > > memtable
>> > > > > > (buffer), flushing, system recovery, and memtable management.
>> > > > > >   - Query : The query process in IoTDB, including multiple
>> types of
>> > > > > > queries, such as raw data query, aggregations, group by time,
>> group
>> > > by
>> > > > > > device.
>> > > > > >   - Merge : Compaction of data files, including multiple
>> compaction
>> > > > > > strategies, such as (1) merging unsequence TsFile with sequence
>> > > TsFile,
>> > > > > (2)
>> > > > > > merging small sequence TsFiles into a large TsFile.
>> > > > > >   - Sync: Synchronize data files from one IoTDB instance to
>> > another.
>> > > > > >   - Authority: Administration of IoTDB, including users, roles
>> and
>> > > > > > privileges.
>> > > > > >   - WAL : Write-ahead-log.
>> > > > > >   - Client: Rpc between client and server, a command-line
>> interface
>> > > and
>> > > > > > client of different languages.
>> > > > > >   - Distribution : The shared-nothing distribution of IoTDB,
>> > > including
>> > > > > raft
>> > > > > > protocol, data partition, and system information management.
>> > > > > >   - Grafana : The connector for using IoTDB as a data source in
>> > > > Grafana.
>> > > > > >   - Spark-TsFile : A connector that allows users read/write
>> TsFile
>> > > > > through
>> > > > > > SparkSQL (treat TsFile as an external data source for SparkSQL).
>> > > > > >   - Spark-IoTDB : A connector that allows users read/write IoTDB
>> > > > through
>> > > > > > SparkSQL (treat IoTDB as an external data source for SparkSQL).
>> > > > > >   - Hive-tsfile : A connector that maps TsFile as a table in
>> Hive
>> > for
>> > > > the
>> > > > > > query.
>> > > > > >   - Hadoop-tsfile : A connector that allows users read/write
>> TsFile
>> > > > > through
>> > > > > > MapReduce.
>> > > > > >   - Tools : Other tools for IoTDB, such as watermark, memory
>> > > > estimation,
>> > > > > > JMX, query history visualization.
>> > > > > >   - Example : Examples of different modules, such as JDBC,
>> Session,
>> > > > > TsFile.
>> > > > > >   - WebSite : IoTDB official website.
>> > > > > >   - Doc: documents of IoTDB, docs folder under root directory.
>> > > > > >   - Other: Those do not belong to the above components.
>> > > > > >
>> > > > > >
>> > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午5:28写道：
>> > > > > >
>> > > > > > > Hi,
>> > > > > > >
>> > > > > > > @Xiangdong thank you for helping to clarify my original
>> > intention.
>> > > > you
>> > > > > > are
>> > > > > > > right, I really want to list some components to let users
>> choose
>> > > > which
>> > > > > > > component the given issue belongs to.
>> > > > > > >
>> > > > > > > @Jialin I think approach 4 would be more meaningful for IoTDB.
>> > > Could
>> > > > > you
>> > > > > > > please add the detailed description for each component, such
>> as,
>> > > > > describe
>> > > > > > > the relationship between `tsfile` and
>> > > > > > > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which
>> > > changes
>> > > > > > belong
>> > > > > > > to `Other` component, etc., which can help us better
>> understand
>> > the
>> > > > > > reasons
>> > > > > > > for such classification. On the other hand, we will further
>> > discuss
>> > > > > > whether
>> > > > > > > there is optimization space for this component classification.
>> > > > > > >
>> > > > > > > What do you think?
>> > > > > > >
>> > > > > > > Best,
>> > > > > > > Jincheng
>> > > > > > >
>> > > > > > > Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
>> > > > > > >
>> > > > > > > > >  No that is not the Apache Way. No one should be in
>> charge of
>> > > an
>> > > > > area
>> > > > > > > >
>> > > > > > > > I think Jincheng's mean is, we can list some components to
>> let
>> > > > users
>> > > > > > > > choose which component the given issue belongs to.
>> > > > > > > >
>> > > > > > > > As for which components should be listed, we need to have a
>> > > > > discussion.
>> > > > > > > >
>> > > > > > > > > No one should be in charge of an area
>> > > > > > > >
>> > > > > > > > Yes everyone can assign an issue to himself/herself.
>> > > > > > > >
>> > > > > > > > Best,
>> > > > > > > > ----------------------------------
>> > > > > > > > Xiangdong Huang
>> > > > > > > > School of Software, Tsinghua University
>> > > > > > > >
>> > > > > > > >  黄向东
>> > > > > > > > 清华大学 软件学院
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四
>> 下午4:38写道：
>> > > > > > > >
>> > > > > > > > > > No that is not the Apache Way. No one should be in
>> charge
>> > of
>> > > an
>> > > > > > area.
>> > > > > > > > >
>> > > > > > > > > Oh, great thanks for correcting me Justin, maybe it's my
>> > > language
>> > > > > > > > > expression problems. I mean some contributors and
>> committers
>> > > will
>> > > > > be
>> > > > > > > very
>> > > > > > > > > interested in some components, then spend more time, and
>> be
>> > > more
>> > > > > > > familiar
>> > > > > > > > > with the code.
>> > > > > > > > >
>> > > > > > > > > Best,
>> > > > > > > > > Jincheng
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四
>> > > 下午4:27写道：
>> > > > > > > > >
>> > > > > > > > > > Hi,
>> > > > > > > > > >
>> > > > > > > > > > > When our community grows stronger, each\ module may
>> have
>> > > some
>> > > > > > > > specific
>> > > > > > > > > > person in charge.
>> > > > > > > > > >
>> > > > > > > > > > No that is not the Apache Way. No one should be in
>> charge
>> > of
>> > > an
>> > > > > > area.
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > > Justin
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > —————————————————
>> > > > > > Jialin Qiao
>> > > > > > School of Software, Tsinghua University
>> > > > > >
>> > > > > > 乔嘉林
>> > > > > > 清华大学 软件学院
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > —————————————————
>> > > > Jialin Qiao
>> > > > School of Software, Tsinghua University
>> > > >
>> > > > 乔嘉林
>> > > > 清华大学 软件学院
>> > > >
>> > >
>> >
>> >
>> > --
>> > —————————————————
>> > Jialin Qiao
>> > School of Software, Tsinghua University
>> >
>> > 乔嘉林
>> > 清华大学 软件学院
>> >
>>
>

Re: [DISCUSS] Add the components definition in JIRA template

Posted by jincheng sun <su...@gmail.com>.

Great thanks for adding more relationship info about `Merge`  and
`Storage`.  @Xiangdong !

As :
> (now query engine is in the storage engine..) Merge is a submodule of
storage engine and query engine.

I think `query engine` and `storage engine` is make sense to me. Here only
a suggestion that how about use the `Engine` for both `Storage Engine` and
`Query Engine` as they are very close each other and they also have the
common share `Merge`. i.e.,:

--------
Core
Core/TsFile
Core/Engine
Core/WAL
---------------

What to do you think?

Best,
Jincheng


------------------------------------------------------
1. --Connectors-------------
Connectors components include all data integration with external systems,
The components are listed as follows:

Connectors
Connectors/Grafana
Connectors/Spark-TsFile
Connectors/Spark-IoTDB
Connectors/Hive-tsfile
Connectors/Hadoop-tsfile

2. --Client-------------
Client components involves RPC between client and server, a command-line
interface（multi-languages).The components are listed as follows:

Client
Client/Java
Client/Python

3. --Planner-------------
Planner components involves the query process in IoTDB, including multiple
types of queries, such as raw data query, aggregations, group by time,
group by device, and includes many optimized rules.The components are
listed as follows:

Planner
Planner/sql parser
Planner/sql optimizer

4. --Core-------------
Core components is the storage engine of IoTDB. The components are listed
as follows:

Core
Core/TsFile
Core/Engine
Core/WAL

5. --Tools-------------
Tools components involves many utils of IoTDB, Such as data Sync. The
components are listed as follows:
Tools
Tools/Sync
Tools/Other

6. --Authority---------
Authority

7. --Distribution------
Distribution

8. --Example-----------
Example

9. --WebSite-----------
WebSite

10. --Doc---------------
Doc

11. --Other------------
Other （I'm not sure if we really need the Other component, but there is no
harm in adding one）

Xiangdong Huang <sa...@gmail.com> 于2020年1月17日周五 上午12:50写道：

> Hi,
>
> Thanks jincheng and jialin.
>
> Sorry for absence of the discussion because of  busy work these days..
>
> Some comments:
>
> > Core
> > Core/TsFile
> > Core/Merge
> > Core/Storage
> > Core/WAL
>
> Not so clearly.. maybe TsFile, WAL, Storage Engine, Query Engine? (now
> query engine is in the storage engine..)
> Merge is a submodule of storage engine and query engine.
> If we pick out the merge module, we also need to pick out the FLUSH,
> MODIFICATION, etc.. I think.
>
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Jialin Qiao <qi...@apache.org> 于2020年1月16日周四 下午11:21写道：
>
> > Hi Jincheng,
> >
> > Thank you! Looks good :)
> >
> > jincheng sun <su...@gmail.com> 于2020年1月16日周四 下午4:01写道：
> >
> > > Thanks for your explanation! @Jialin
> > >
> > > I thought about the composition of our components from the external
> > system,
> > > the user level, to the IoTDB bottom-level implementation to the
> auxiliary
> > > tools, and I adjusted a little based on yours.  as follows:
> > >
> > > 1. --Connectors-------------
> > > Connectors components include all data integration with external
> systems,
> > > The components are listed as follows:
> > >
> > > Connectors
> > > Connectors/Grafana
> > > Connectors/Spark-TsFile
> > > Connectors/Spark-IoTDB
> > > Connectors/Hive-tsfile
> > > Connectors/Hadoop-tsfile
> > >
> > > 2. --Client-------------
> > > Client components involves RPC between client and server, a
> command-line
> > > interface（multi-languages).The components are listed as follows:
> > >
> > > Client
> > > Client/Java
> > > Client/Python
> > >
> > > 3. --Planner-------------
> > > Planner components involves the query process in IoTDB, including
> > multiple
> > > types of queries, such as raw data query, aggregations, group by time,
> > > group by device, and includes many optimized rules.The components are
> > > listed as follows:
> > >
> > > Planner
> > > Planner/sql parsar
> > > Planner/sql optimizer
> > >
> > > 4. --Core-------------
> > > Core components is the storage engine of IoTDB. The components are
> listed
> > > as follows:
> > >
> > > Core
> > > Core/TsFile
> > > Core/Merge
> > > Core/Storage
> > > Core/WAL
> > >
> > > 5. --Tools-------------
> > > Tools components involves many utils of IoTDB, Such as data Sync. The
> > > components are listed as follows:
> > > Tools
> > > Tools/Sync
> > > Tools/Other
> > >
> > > 6. --Authority---------
> > > Authority
> > >
> > > 7. --Distribution------
> > > Distribution
> > >
> > > 8. --Example-----------
> > > Example
> > >
> > > 9. --WebSite-----------
> > > WebSite
> > >
> > > 10. --Doc---------------
> > > Doc
> > >
> > > 11. --Other------------
> > > Other （I'm not sure if we really need the Other component, but there is
> > no
> > > harm in adding one）
> > >
> > > This does not change much from your solution. The core is to add a
> layer
> > of
> > > abstraction.
> > >
> > > Does this make sense to you? Anyway we can continue to improve if
> needed!
> > >
> > > Best,
> > > Jincheng
> > >
> > > Jialin Qiao <qi...@apache.org> 于2020年1月15日周三 下午4:28写道：
> > >
> > > > Hi,
> > > >
> > > > WAL has an independent physical layout that has nothing to do with
> > > TsFile.
> > > > It is used in the writing process and recovering in IoTDB.
> > > > It could be merged into Storage components.
> > > >
> > > > I'm not sure whether I leave out some components, so I add an Other
> and
> > > > therefore can not give an example...
> > > >
> > > > Thanks,
> > > >
> > > > jincheng sun <su...@gmail.com> 于2020年1月15日周三 下午4:01写道：
> > > >
> > > > > Sorry for the late reply, I just came back from team outing.
> > > > >
> > > > > Thanks for adding more detail information. I have two questions
> then:
> > > > >
> > > > > - Is WAL a write strategy for TsFlie, can WAL be incorporated into
> > > other
> > > > > components and what is the reason for being a separate component?
> > > > > - Can you give an example of what changes belongs to the Other
> > > component?
> > > > >
> > > > > Best,
> > > > > Jincheng
> > > > >
> > > > >
> > > > > Jialin Qiao <qi...@apache.org> 于2020年1月9日周四 下午6:39写道：
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Sure, maybe a bit more...
> > > > > >
> > > > > >   - TsFile : A columnar file format that has flexible schema and
> > data
> > > > > > management for time series.
> > > > > >   - Storage : The storage engine of IoTDB, including writing to
> > > > memtable
> > > > > > (buffer), flushing, system recovery, and memtable management.
> > > > > >   - Query : The query process in IoTDB, including multiple types
> of
> > > > > > queries, such as raw data query, aggregations, group by time,
> group
> > > by
> > > > > > device.
> > > > > >   - Merge : Compaction of data files, including multiple
> compaction
> > > > > > strategies, such as (1) merging unsequence TsFile with sequence
> > > TsFile,
> > > > > (2)
> > > > > > merging small sequence TsFiles into a large TsFile.
> > > > > >   - Sync: Synchronize data files from one IoTDB instance to
> > another.
> > > > > >   - Authority: Administration of IoTDB, including users, roles
> and
> > > > > > privileges.
> > > > > >   - WAL : Write-ahead-log.
> > > > > >   - Client: Rpc between client and server, a command-line
> interface
> > > and
> > > > > > client of different languages.
> > > > > >   - Distribution : The shared-nothing distribution of IoTDB,
> > > including
> > > > > raft
> > > > > > protocol, data partition, and system information management.
> > > > > >   - Grafana : The connector for using IoTDB as a data source in
> > > > Grafana.
> > > > > >   - Spark-TsFile : A connector that allows users read/write
> TsFile
> > > > > through
> > > > > > SparkSQL (treat TsFile as an external data source for SparkSQL).
> > > > > >   - Spark-IoTDB : A connector that allows users read/write IoTDB
> > > > through
> > > > > > SparkSQL (treat IoTDB as an external data source for SparkSQL).
> > > > > >   - Hive-tsfile : A connector that maps TsFile as a table in Hive
> > for
> > > > the
> > > > > > query.
> > > > > >   - Hadoop-tsfile : A connector that allows users read/write
> TsFile
> > > > > through
> > > > > > MapReduce.
> > > > > >   - Tools : Other tools for IoTDB, such as watermark, memory
> > > > estimation,
> > > > > > JMX, query history visualization.
> > > > > >   - Example : Examples of different modules, such as JDBC,
> Session,
> > > > > TsFile.
> > > > > >   - WebSite : IoTDB official website.
> > > > > >   - Doc: documents of IoTDB, docs folder under root directory.
> > > > > >   - Other: Those do not belong to the above components.
> > > > > >
> > > > > >
> > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午5:28写道：
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > @Xiangdong thank you for helping to clarify my original
> > intention.
> > > > you
> > > > > > are
> > > > > > > right, I really want to list some components to let users
> choose
> > > > which
> > > > > > > component the given issue belongs to.
> > > > > > >
> > > > > > > @Jialin I think approach 4 would be more meaningful for IoTDB.
> > > Could
> > > > > you
> > > > > > > please add the detailed description for each component, such
> as,
> > > > > describe
> > > > > > > the relationship between `tsfile` and
> > > > > > > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which
> > > changes
> > > > > > belong
> > > > > > > to `Other` component, etc., which can help us better understand
> > the
> > > > > > reasons
> > > > > > > for such classification. On the other hand, we will further
> > discuss
> > > > > > whether
> > > > > > > there is optimization space for this component classification.
> > > > > > >
> > > > > > > What do you think?
> > > > > > >
> > > > > > > Best,
> > > > > > > Jincheng
> > > > > > >
> > > > > > > Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
> > > > > > >
> > > > > > > > >  No that is not the Apache Way. No one should be in charge
> of
> > > an
> > > > > area
> > > > > > > >
> > > > > > > > I think Jincheng's mean is, we can list some components to
> let
> > > > users
> > > > > > > > choose which component the given issue belongs to.
> > > > > > > >
> > > > > > > > As for which components should be listed, we need to have a
> > > > > discussion.
> > > > > > > >
> > > > > > > > > No one should be in charge of an area
> > > > > > > >
> > > > > > > > Yes everyone can assign an issue to himself/herself.
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > ----------------------------------
> > > > > > > > Xiangdong Huang
> > > > > > > > School of Software, Tsinghua University
> > > > > > > >
> > > > > > > >  黄向东
> > > > > > > > 清华大学 软件学院
> > > > > > > >
> > > > > > > >
> > > > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四
> 下午4:38写道：
> > > > > > > >
> > > > > > > > > > No that is not the Apache Way. No one should be in charge
> > of
> > > an
> > > > > > area.
> > > > > > > > >
> > > > > > > > > Oh, great thanks for correcting me Justin, maybe it's my
> > > language
> > > > > > > > > expression problems. I mean some contributors and
> committers
> > > will
> > > > > be
> > > > > > > very
> > > > > > > > > interested in some components, then spend more time, and be
> > > more
> > > > > > > familiar
> > > > > > > > > with the code.
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Jincheng
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四
> > > 下午4:27写道：
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > > When our community grows stronger, each\ module may
> have
> > > some
> > > > > > > > specific
> > > > > > > > > > person in charge.
> > > > > > > > > >
> > > > > > > > > > No that is not the Apache Way. No one should be in charge
> > of
> > > an
> > > > > > area.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Justin
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > —————————————————
> > > > > > Jialin Qiao
> > > > > > School of Software, Tsinghua University
> > > > > >
> > > > > > 乔嘉林
> > > > > > 清华大学 软件学院
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > —————————————————
> > > > Jialin Qiao
> > > > School of Software, Tsinghua University
> > > >
> > > > 乔嘉林
> > > > 清华大学 软件学院
> > > >
> > >
> >
> >
> > --
> > —————————————————
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
>

Re: [DISCUSS] Add the components definition in JIRA template

Posted by Xiangdong Huang <sa...@gmail.com>.

Hi,

Thanks jincheng and jialin.

Sorry for absence of the discussion because of  busy work these days..

Some comments:

> Core
> Core/TsFile
> Core/Merge
> Core/Storage
> Core/WAL

Not so clearly.. maybe TsFile, WAL, Storage Engine, Query Engine? (now
query engine is in the storage engine..)
Merge is a submodule of storage engine and query engine.
If we pick out the merge module, we also need to pick out the FLUSH,
MODIFICATION, etc.. I think.

Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Jialin Qiao <qi...@apache.org> 于2020年1月16日周四 下午11:21写道：

> Hi Jincheng,
>
> Thank you! Looks good :)
>
> jincheng sun <su...@gmail.com> 于2020年1月16日周四 下午4:01写道：
>
> > Thanks for your explanation! @Jialin
> >
> > I thought about the composition of our components from the external
> system,
> > the user level, to the IoTDB bottom-level implementation to the auxiliary
> > tools, and I adjusted a little based on yours.  as follows:
> >
> > 1. --Connectors-------------
> > Connectors components include all data integration with external systems,
> > The components are listed as follows:
> >
> > Connectors
> > Connectors/Grafana
> > Connectors/Spark-TsFile
> > Connectors/Spark-IoTDB
> > Connectors/Hive-tsfile
> > Connectors/Hadoop-tsfile
> >
> > 2. --Client-------------
> > Client components involves RPC between client and server, a command-line
> > interface（multi-languages).The components are listed as follows:
> >
> > Client
> > Client/Java
> > Client/Python
> >
> > 3. --Planner-------------
> > Planner components involves the query process in IoTDB, including
> multiple
> > types of queries, such as raw data query, aggregations, group by time,
> > group by device, and includes many optimized rules.The components are
> > listed as follows:
> >
> > Planner
> > Planner/sql parsar
> > Planner/sql optimizer
> >
> > 4. --Core-------------
> > Core components is the storage engine of IoTDB. The components are listed
> > as follows:
> >
> > Core
> > Core/TsFile
> > Core/Merge
> > Core/Storage
> > Core/WAL
> >
> > 5. --Tools-------------
> > Tools components involves many utils of IoTDB, Such as data Sync. The
> > components are listed as follows:
> > Tools
> > Tools/Sync
> > Tools/Other
> >
> > 6. --Authority---------
> > Authority
> >
> > 7. --Distribution------
> > Distribution
> >
> > 8. --Example-----------
> > Example
> >
> > 9. --WebSite-----------
> > WebSite
> >
> > 10. --Doc---------------
> > Doc
> >
> > 11. --Other------------
> > Other （I'm not sure if we really need the Other component, but there is
> no
> > harm in adding one）
> >
> > This does not change much from your solution. The core is to add a layer
> of
> > abstraction.
> >
> > Does this make sense to you? Anyway we can continue to improve if needed!
> >
> > Best,
> > Jincheng
> >
> > Jialin Qiao <qi...@apache.org> 于2020年1月15日周三 下午4:28写道：
> >
> > > Hi,
> > >
> > > WAL has an independent physical layout that has nothing to do with
> > TsFile.
> > > It is used in the writing process and recovering in IoTDB.
> > > It could be merged into Storage components.
> > >
> > > I'm not sure whether I leave out some components, so I add an Other and
> > > therefore can not give an example...
> > >
> > > Thanks,
> > >
> > > jincheng sun <su...@gmail.com> 于2020年1月15日周三 下午4:01写道：
> > >
> > > > Sorry for the late reply, I just came back from team outing.
> > > >
> > > > Thanks for adding more detail information. I have two questions then:
> > > >
> > > > - Is WAL a write strategy for TsFlie, can WAL be incorporated into
> > other
> > > > components and what is the reason for being a separate component?
> > > > - Can you give an example of what changes belongs to the Other
> > component?
> > > >
> > > > Best,
> > > > Jincheng
> > > >
> > > >
> > > > Jialin Qiao <qi...@apache.org> 于2020年1月9日周四 下午6:39写道：
> > > >
> > > > > Hi,
> > > > >
> > > > > Sure, maybe a bit more...
> > > > >
> > > > >   - TsFile : A columnar file format that has flexible schema and
> data
> > > > > management for time series.
> > > > >   - Storage : The storage engine of IoTDB, including writing to
> > > memtable
> > > > > (buffer), flushing, system recovery, and memtable management.
> > > > >   - Query : The query process in IoTDB, including multiple types of
> > > > > queries, such as raw data query, aggregations, group by time, group
> > by
> > > > > device.
> > > > >   - Merge : Compaction of data files, including multiple compaction
> > > > > strategies, such as (1) merging unsequence TsFile with sequence
> > TsFile,
> > > > (2)
> > > > > merging small sequence TsFiles into a large TsFile.
> > > > >   - Sync: Synchronize data files from one IoTDB instance to
> another.
> > > > >   - Authority: Administration of IoTDB, including users, roles and
> > > > > privileges.
> > > > >   - WAL : Write-ahead-log.
> > > > >   - Client: Rpc between client and server, a command-line interface
> > and
> > > > > client of different languages.
> > > > >   - Distribution : The shared-nothing distribution of IoTDB,
> > including
> > > > raft
> > > > > protocol, data partition, and system information management.
> > > > >   - Grafana : The connector for using IoTDB as a data source in
> > > Grafana.
> > > > >   - Spark-TsFile : A connector that allows users read/write TsFile
> > > > through
> > > > > SparkSQL (treat TsFile as an external data source for SparkSQL).
> > > > >   - Spark-IoTDB : A connector that allows users read/write IoTDB
> > > through
> > > > > SparkSQL (treat IoTDB as an external data source for SparkSQL).
> > > > >   - Hive-tsfile : A connector that maps TsFile as a table in Hive
> for
> > > the
> > > > > query.
> > > > >   - Hadoop-tsfile : A connector that allows users read/write TsFile
> > > > through
> > > > > MapReduce.
> > > > >   - Tools : Other tools for IoTDB, such as watermark, memory
> > > estimation,
> > > > > JMX, query history visualization.
> > > > >   - Example : Examples of different modules, such as JDBC, Session,
> > > > TsFile.
> > > > >   - WebSite : IoTDB official website.
> > > > >   - Doc: documents of IoTDB, docs folder under root directory.
> > > > >   - Other: Those do not belong to the above components.
> > > > >
> > > > >
> > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午5:28写道：
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > @Xiangdong thank you for helping to clarify my original
> intention.
> > > you
> > > > > are
> > > > > > right, I really want to list some components to let users choose
> > > which
> > > > > > component the given issue belongs to.
> > > > > >
> > > > > > @Jialin I think approach 4 would be more meaningful for IoTDB.
> > Could
> > > > you
> > > > > > please add the detailed description for each component, such as,
> > > > describe
> > > > > > the relationship between `tsfile` and
> > > > > > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which
> > changes
> > > > > belong
> > > > > > to `Other` component, etc., which can help us better understand
> the
> > > > > reasons
> > > > > > for such classification. On the other hand, we will further
> discuss
> > > > > whether
> > > > > > there is optimization space for this component classification.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > Best,
> > > > > > Jincheng
> > > > > >
> > > > > > Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
> > > > > >
> > > > > > > >  No that is not the Apache Way. No one should be in charge of
> > an
> > > > area
> > > > > > >
> > > > > > > I think Jincheng's mean is, we can list some components to let
> > > users
> > > > > > > choose which component the given issue belongs to.
> > > > > > >
> > > > > > > As for which components should be listed, we need to have a
> > > > discussion.
> > > > > > >
> > > > > > > > No one should be in charge of an area
> > > > > > >
> > > > > > > Yes everyone can assign an issue to himself/herself.
> > > > > > >
> > > > > > > Best,
> > > > > > > ----------------------------------
> > > > > > > Xiangdong Huang
> > > > > > > School of Software, Tsinghua University
> > > > > > >
> > > > > > >  黄向东
> > > > > > > 清华大学 软件学院
> > > > > > >
> > > > > > >
> > > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午4:38写道：
> > > > > > >
> > > > > > > > > No that is not the Apache Way. No one should be in charge
> of
> > an
> > > > > area.
> > > > > > > >
> > > > > > > > Oh, great thanks for correcting me Justin, maybe it's my
> > language
> > > > > > > > expression problems. I mean some contributors and committers
> > will
> > > > be
> > > > > > very
> > > > > > > > interested in some components, then spend more time, and be
> > more
> > > > > > familiar
> > > > > > > > with the code.
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Jincheng
> > > > > > > >
> > > > > > > >
> > > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四
> > 下午4:27写道：
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > > When our community grows stronger, each\ module may have
> > some
> > > > > > > specific
> > > > > > > > > person in charge.
> > > > > > > > >
> > > > > > > > > No that is not the Apache Way. No one should be in charge
> of
> > an
> > > > > area.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Justin
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > —————————————————
> > > > > Jialin Qiao
> > > > > School of Software, Tsinghua University
> > > > >
> > > > > 乔嘉林
> > > > > 清华大学 软件学院
> > > > >
> > > >
> > >
> > >
> > > --
> > > —————————————————
> > > Jialin Qiao
> > > School of Software, Tsinghua University
> > >
> > > 乔嘉林
> > > 清华大学 软件学院
> > >
> >
>
>
> --
> —————————————————
> Jialin Qiao
> School of Software, Tsinghua University
>
> 乔嘉林
> 清华大学 软件学院
>

Re: [DISCUSS] Add the components definition in JIRA template

Posted by Jialin Qiao <qi...@apache.org>.

Hi Jincheng,

Thank you! Looks good :)

jincheng sun <su...@gmail.com> 于2020年1月16日周四 下午4:01写道：

> Thanks for your explanation! @Jialin
>
> I thought about the composition of our components from the external system,
> the user level, to the IoTDB bottom-level implementation to the auxiliary
> tools, and I adjusted a little based on yours.  as follows:
>
> 1. --Connectors-------------
> Connectors components include all data integration with external systems,
> The components are listed as follows:
>
> Connectors
> Connectors/Grafana
> Connectors/Spark-TsFile
> Connectors/Spark-IoTDB
> Connectors/Hive-tsfile
> Connectors/Hadoop-tsfile
>
> 2. --Client-------------
> Client components involves RPC between client and server, a command-line
> interface（multi-languages).The components are listed as follows:
>
> Client
> Client/Java
> Client/Python
>
> 3. --Planner-------------
> Planner components involves the query process in IoTDB, including multiple
> types of queries, such as raw data query, aggregations, group by time,
> group by device, and includes many optimized rules.The components are
> listed as follows:
>
> Planner
> Planner/sql parsar
> Planner/sql optimizer
>
> 4. --Core-------------
> Core components is the storage engine of IoTDB. The components are listed
> as follows:
>
> Core
> Core/TsFile
> Core/Merge
> Core/Storage
> Core/WAL
>
> 5. --Tools-------------
> Tools components involves many utils of IoTDB, Such as data Sync. The
> components are listed as follows:
> Tools
> Tools/Sync
> Tools/Other
>
> 6. --Authority---------
> Authority
>
> 7. --Distribution------
> Distribution
>
> 8. --Example-----------
> Example
>
> 9. --WebSite-----------
> WebSite
>
> 10. --Doc---------------
> Doc
>
> 11. --Other------------
> Other （I'm not sure if we really need the Other component, but there is no
> harm in adding one）
>
> This does not change much from your solution. The core is to add a layer of
> abstraction.
>
> Does this make sense to you? Anyway we can continue to improve if needed!
>
> Best,
> Jincheng
>
> Jialin Qiao <qi...@apache.org> 于2020年1月15日周三 下午4:28写道：
>
> > Hi,
> >
> > WAL has an independent physical layout that has nothing to do with
> TsFile.
> > It is used in the writing process and recovering in IoTDB.
> > It could be merged into Storage components.
> >
> > I'm not sure whether I leave out some components, so I add an Other and
> > therefore can not give an example...
> >
> > Thanks,
> >
> > jincheng sun <su...@gmail.com> 于2020年1月15日周三 下午4:01写道：
> >
> > > Sorry for the late reply, I just came back from team outing.
> > >
> > > Thanks for adding more detail information. I have two questions then:
> > >
> > > - Is WAL a write strategy for TsFlie, can WAL be incorporated into
> other
> > > components and what is the reason for being a separate component?
> > > - Can you give an example of what changes belongs to the Other
> component?
> > >
> > > Best,
> > > Jincheng
> > >
> > >
> > > Jialin Qiao <qi...@apache.org> 于2020年1月9日周四 下午6:39写道：
> > >
> > > > Hi,
> > > >
> > > > Sure, maybe a bit more...
> > > >
> > > >   - TsFile : A columnar file format that has flexible schema and data
> > > > management for time series.
> > > >   - Storage : The storage engine of IoTDB, including writing to
> > memtable
> > > > (buffer), flushing, system recovery, and memtable management.
> > > >   - Query : The query process in IoTDB, including multiple types of
> > > > queries, such as raw data query, aggregations, group by time, group
> by
> > > > device.
> > > >   - Merge : Compaction of data files, including multiple compaction
> > > > strategies, such as (1) merging unsequence TsFile with sequence
> TsFile,
> > > (2)
> > > > merging small sequence TsFiles into a large TsFile.
> > > >   - Sync: Synchronize data files from one IoTDB instance to another.
> > > >   - Authority: Administration of IoTDB, including users, roles and
> > > > privileges.
> > > >   - WAL : Write-ahead-log.
> > > >   - Client: Rpc between client and server, a command-line interface
> and
> > > > client of different languages.
> > > >   - Distribution : The shared-nothing distribution of IoTDB,
> including
> > > raft
> > > > protocol, data partition, and system information management.
> > > >   - Grafana : The connector for using IoTDB as a data source in
> > Grafana.
> > > >   - Spark-TsFile : A connector that allows users read/write TsFile
> > > through
> > > > SparkSQL (treat TsFile as an external data source for SparkSQL).
> > > >   - Spark-IoTDB : A connector that allows users read/write IoTDB
> > through
> > > > SparkSQL (treat IoTDB as an external data source for SparkSQL).
> > > >   - Hive-tsfile : A connector that maps TsFile as a table in Hive for
> > the
> > > > query.
> > > >   - Hadoop-tsfile : A connector that allows users read/write TsFile
> > > through
> > > > MapReduce.
> > > >   - Tools : Other tools for IoTDB, such as watermark, memory
> > estimation,
> > > > JMX, query history visualization.
> > > >   - Example : Examples of different modules, such as JDBC, Session,
> > > TsFile.
> > > >   - WebSite : IoTDB official website.
> > > >   - Doc: documents of IoTDB, docs folder under root directory.
> > > >   - Other: Those do not belong to the above components.
> > > >
> > > >
> > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午5:28写道：
> > > >
> > > > > Hi,
> > > > >
> > > > > @Xiangdong thank you for helping to clarify my original intention.
> > you
> > > > are
> > > > > right, I really want to list some components to let users choose
> > which
> > > > > component the given issue belongs to.
> > > > >
> > > > > @Jialin I think approach 4 would be more meaningful for IoTDB.
> Could
> > > you
> > > > > please add the detailed description for each component, such as,
> > > describe
> > > > > the relationship between `tsfile` and
> > > > > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which
> changes
> > > > belong
> > > > > to `Other` component, etc., which can help us better understand the
> > > > reasons
> > > > > for such classification. On the other hand, we will further discuss
> > > > whether
> > > > > there is optimization space for this component classification.
> > > > >
> > > > > What do you think?
> > > > >
> > > > > Best,
> > > > > Jincheng
> > > > >
> > > > > Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
> > > > >
> > > > > > >  No that is not the Apache Way. No one should be in charge of
> an
> > > area
> > > > > >
> > > > > > I think Jincheng's mean is, we can list some components to let
> > users
> > > > > > choose which component the given issue belongs to.
> > > > > >
> > > > > > As for which components should be listed, we need to have a
> > > discussion.
> > > > > >
> > > > > > > No one should be in charge of an area
> > > > > >
> > > > > > Yes everyone can assign an issue to himself/herself.
> > > > > >
> > > > > > Best,
> > > > > > ----------------------------------
> > > > > > Xiangdong Huang
> > > > > > School of Software, Tsinghua University
> > > > > >
> > > > > >  黄向东
> > > > > > 清华大学 软件学院
> > > > > >
> > > > > >
> > > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午4:38写道：
> > > > > >
> > > > > > > > No that is not the Apache Way. No one should be in charge of
> an
> > > > area.
> > > > > > >
> > > > > > > Oh, great thanks for correcting me Justin, maybe it's my
> language
> > > > > > > expression problems. I mean some contributors and committers
> will
> > > be
> > > > > very
> > > > > > > interested in some components, then spend more time, and be
> more
> > > > > familiar
> > > > > > > with the code.
> > > > > > >
> > > > > > > Best,
> > > > > > > Jincheng
> > > > > > >
> > > > > > >
> > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四
> 下午4:27写道：
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > > When our community grows stronger, each\ module may have
> some
> > > > > > specific
> > > > > > > > person in charge.
> > > > > > > >
> > > > > > > > No that is not the Apache Way. No one should be in charge of
> an
> > > > area.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Justin
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > —————————————————
> > > > Jialin Qiao
> > > > School of Software, Tsinghua University
> > > >
> > > > 乔嘉林
> > > > 清华大学 软件学院
> > > >
> > >
> >
> >
> > --
> > —————————————————
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
>


-- 
—————————————————
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

Re: [DISCUSS] Add the components definition in JIRA template

Posted by jincheng sun <su...@gmail.com>.

Thanks for your explanation! @Jialin

I thought about the composition of our components from the external system,
the user level, to the IoTDB bottom-level implementation to the auxiliary
tools, and I adjusted a little based on yours.  as follows:

1. --Connectors-------------
Connectors components include all data integration with external systems,
The components are listed as follows:

Connectors
Connectors/Grafana
Connectors/Spark-TsFile
Connectors/Spark-IoTDB
Connectors/Hive-tsfile
Connectors/Hadoop-tsfile

2. --Client-------------
Client components involves RPC between client and server, a command-line
interface（multi-languages).The components are listed as follows:

Client
Client/Java
Client/Python

3. --Planner-------------
Planner components involves the query process in IoTDB, including multiple
types of queries, such as raw data query, aggregations, group by time,
group by device, and includes many optimized rules.The components are
listed as follows:

Planner
Planner/sql parsar
Planner/sql optimizer

4. --Core-------------
Core components is the storage engine of IoTDB. The components are listed
as follows:

Core
Core/TsFile
Core/Merge
Core/Storage
Core/WAL

5. --Tools-------------
Tools components involves many utils of IoTDB, Such as data Sync. The
components are listed as follows:
Tools
Tools/Sync
Tools/Other

6. --Authority---------
Authority

7. --Distribution------
Distribution

8. --Example-----------
Example

9. --WebSite-----------
WebSite

10. --Doc---------------
Doc

11. --Other------------
Other （I'm not sure if we really need the Other component, but there is no
harm in adding one）

This does not change much from your solution. The core is to add a layer of
abstraction.

Does this make sense to you? Anyway we can continue to improve if needed!

Best,
Jincheng

Jialin Qiao <qi...@apache.org> 于2020年1月15日周三 下午4:28写道：

> Hi,
>
> WAL has an independent physical layout that has nothing to do with TsFile.
> It is used in the writing process and recovering in IoTDB.
> It could be merged into Storage components.
>
> I'm not sure whether I leave out some components, so I add an Other and
> therefore can not give an example...
>
> Thanks,
>
> jincheng sun <su...@gmail.com> 于2020年1月15日周三 下午4:01写道：
>
> > Sorry for the late reply, I just came back from team outing.
> >
> > Thanks for adding more detail information. I have two questions then:
> >
> > - Is WAL a write strategy for TsFlie, can WAL be incorporated into other
> > components and what is the reason for being a separate component?
> > - Can you give an example of what changes belongs to the Other component?
> >
> > Best,
> > Jincheng
> >
> >
> > Jialin Qiao <qi...@apache.org> 于2020年1月9日周四 下午6:39写道：
> >
> > > Hi,
> > >
> > > Sure, maybe a bit more...
> > >
> > >   - TsFile : A columnar file format that has flexible schema and data
> > > management for time series.
> > >   - Storage : The storage engine of IoTDB, including writing to
> memtable
> > > (buffer), flushing, system recovery, and memtable management.
> > >   - Query : The query process in IoTDB, including multiple types of
> > > queries, such as raw data query, aggregations, group by time, group by
> > > device.
> > >   - Merge : Compaction of data files, including multiple compaction
> > > strategies, such as (1) merging unsequence TsFile with sequence TsFile,
> > (2)
> > > merging small sequence TsFiles into a large TsFile.
> > >   - Sync: Synchronize data files from one IoTDB instance to another.
> > >   - Authority: Administration of IoTDB, including users, roles and
> > > privileges.
> > >   - WAL : Write-ahead-log.
> > >   - Client: Rpc between client and server, a command-line interface and
> > > client of different languages.
> > >   - Distribution : The shared-nothing distribution of IoTDB, including
> > raft
> > > protocol, data partition, and system information management.
> > >   - Grafana : The connector for using IoTDB as a data source in
> Grafana.
> > >   - Spark-TsFile : A connector that allows users read/write TsFile
> > through
> > > SparkSQL (treat TsFile as an external data source for SparkSQL).
> > >   - Spark-IoTDB : A connector that allows users read/write IoTDB
> through
> > > SparkSQL (treat IoTDB as an external data source for SparkSQL).
> > >   - Hive-tsfile : A connector that maps TsFile as a table in Hive for
> the
> > > query.
> > >   - Hadoop-tsfile : A connector that allows users read/write TsFile
> > through
> > > MapReduce.
> > >   - Tools : Other tools for IoTDB, such as watermark, memory
> estimation,
> > > JMX, query history visualization.
> > >   - Example : Examples of different modules, such as JDBC, Session,
> > TsFile.
> > >   - WebSite : IoTDB official website.
> > >   - Doc: documents of IoTDB, docs folder under root directory.
> > >   - Other: Those do not belong to the above components.
> > >
> > >
> > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午5:28写道：
> > >
> > > > Hi,
> > > >
> > > > @Xiangdong thank you for helping to clarify my original intention.
> you
> > > are
> > > > right, I really want to list some components to let users choose
> which
> > > > component the given issue belongs to.
> > > >
> > > > @Jialin I think approach 4 would be more meaningful for IoTDB. Could
> > you
> > > > please add the detailed description for each component, such as,
> > describe
> > > > the relationship between `tsfile` and
> > > > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which changes
> > > belong
> > > > to `Other` component, etc., which can help us better understand the
> > > reasons
> > > > for such classification. On the other hand, we will further discuss
> > > whether
> > > > there is optimization space for this component classification.
> > > >
> > > > What do you think?
> > > >
> > > > Best,
> > > > Jincheng
> > > >
> > > > Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
> > > >
> > > > > >  No that is not the Apache Way. No one should be in charge of an
> > area
> > > > >
> > > > > I think Jincheng's mean is, we can list some components to let
> users
> > > > > choose which component the given issue belongs to.
> > > > >
> > > > > As for which components should be listed, we need to have a
> > discussion.
> > > > >
> > > > > > No one should be in charge of an area
> > > > >
> > > > > Yes everyone can assign an issue to himself/herself.
> > > > >
> > > > > Best,
> > > > > ----------------------------------
> > > > > Xiangdong Huang
> > > > > School of Software, Tsinghua University
> > > > >
> > > > >  黄向东
> > > > > 清华大学 软件学院
> > > > >
> > > > >
> > > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午4:38写道：
> > > > >
> > > > > > > No that is not the Apache Way. No one should be in charge of an
> > > area.
> > > > > >
> > > > > > Oh, great thanks for correcting me Justin, maybe it's my language
> > > > > > expression problems. I mean some contributors and committers will
> > be
> > > > very
> > > > > > interested in some components, then spend more time, and be more
> > > > familiar
> > > > > > with the code.
> > > > > >
> > > > > > Best,
> > > > > > Jincheng
> > > > > >
> > > > > >
> > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 下午4:27写道：
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > > When our community grows stronger, each\ module may have some
> > > > > specific
> > > > > > > person in charge.
> > > > > > >
> > > > > > > No that is not the Apache Way. No one should be in charge of an
> > > area.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Justin
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > —————————————————
> > > Jialin Qiao
> > > School of Software, Tsinghua University
> > >
> > > 乔嘉林
> > > 清华大学 软件学院
> > >
> >
>
>
> --
> —————————————————
> Jialin Qiao
> School of Software, Tsinghua University
>
> 乔嘉林
> 清华大学 软件学院
>

Re: [DISCUSS] Add the components definition in JIRA template

Posted by Jialin Qiao <qi...@apache.org>.

Hi,

WAL has an independent physical layout that has nothing to do with TsFile.
It is used in the writing process and recovering in IoTDB.
It could be merged into Storage components.

I'm not sure whether I leave out some components, so I add an Other and
therefore can not give an example...

Thanks,

jincheng sun <su...@gmail.com> 于2020年1月15日周三 下午4:01写道：

> Sorry for the late reply, I just came back from team outing.
>
> Thanks for adding more detail information. I have two questions then:
>
> - Is WAL a write strategy for TsFlie, can WAL be incorporated into other
> components and what is the reason for being a separate component?
> - Can you give an example of what changes belongs to the Other component?
>
> Best,
> Jincheng
>
>
> Jialin Qiao <qi...@apache.org> 于2020年1月9日周四 下午6:39写道：
>
> > Hi,
> >
> > Sure, maybe a bit more...
> >
> >   - TsFile : A columnar file format that has flexible schema and data
> > management for time series.
> >   - Storage : The storage engine of IoTDB, including writing to memtable
> > (buffer), flushing, system recovery, and memtable management.
> >   - Query : The query process in IoTDB, including multiple types of
> > queries, such as raw data query, aggregations, group by time, group by
> > device.
> >   - Merge : Compaction of data files, including multiple compaction
> > strategies, such as (1) merging unsequence TsFile with sequence TsFile,
> (2)
> > merging small sequence TsFiles into a large TsFile.
> >   - Sync: Synchronize data files from one IoTDB instance to another.
> >   - Authority: Administration of IoTDB, including users, roles and
> > privileges.
> >   - WAL : Write-ahead-log.
> >   - Client: Rpc between client and server, a command-line interface and
> > client of different languages.
> >   - Distribution : The shared-nothing distribution of IoTDB, including
> raft
> > protocol, data partition, and system information management.
> >   - Grafana : The connector for using IoTDB as a data source in Grafana.
> >   - Spark-TsFile : A connector that allows users read/write TsFile
> through
> > SparkSQL (treat TsFile as an external data source for SparkSQL).
> >   - Spark-IoTDB : A connector that allows users read/write IoTDB through
> > SparkSQL (treat IoTDB as an external data source for SparkSQL).
> >   - Hive-tsfile : A connector that maps TsFile as a table in Hive for the
> > query.
> >   - Hadoop-tsfile : A connector that allows users read/write TsFile
> through
> > MapReduce.
> >   - Tools : Other tools for IoTDB, such as watermark, memory estimation,
> > JMX, query history visualization.
> >   - Example : Examples of different modules, such as JDBC, Session,
> TsFile.
> >   - WebSite : IoTDB official website.
> >   - Doc: documents of IoTDB, docs folder under root directory.
> >   - Other: Those do not belong to the above components.
> >
> >
> > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午5:28写道：
> >
> > > Hi,
> > >
> > > @Xiangdong thank you for helping to clarify my original intention. you
> > are
> > > right, I really want to list some components to let users choose which
> > > component the given issue belongs to.
> > >
> > > @Jialin I think approach 4 would be more meaningful for IoTDB. Could
> you
> > > please add the detailed description for each component, such as,
> describe
> > > the relationship between `tsfile` and
> > > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which changes
> > belong
> > > to `Other` component, etc., which can help us better understand the
> > reasons
> > > for such classification. On the other hand, we will further discuss
> > whether
> > > there is optimization space for this component classification.
> > >
> > > What do you think?
> > >
> > > Best,
> > > Jincheng
> > >
> > > Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
> > >
> > > > >  No that is not the Apache Way. No one should be in charge of an
> area
> > > >
> > > > I think Jincheng's mean is, we can list some components to let users
> > > > choose which component the given issue belongs to.
> > > >
> > > > As for which components should be listed, we need to have a
> discussion.
> > > >
> > > > > No one should be in charge of an area
> > > >
> > > > Yes everyone can assign an issue to himself/herself.
> > > >
> > > > Best,
> > > > ----------------------------------
> > > > Xiangdong Huang
> > > > School of Software, Tsinghua University
> > > >
> > > >  黄向东
> > > > 清华大学 软件学院
> > > >
> > > >
> > > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午4:38写道：
> > > >
> > > > > > No that is not the Apache Way. No one should be in charge of an
> > area.
> > > > >
> > > > > Oh, great thanks for correcting me Justin, maybe it's my language
> > > > > expression problems. I mean some contributors and committers will
> be
> > > very
> > > > > interested in some components, then spend more time, and be more
> > > familiar
> > > > > with the code.
> > > > >
> > > > > Best,
> > > > > Jincheng
> > > > >
> > > > >
> > > > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 下午4:27写道：
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > > When our community grows stronger, each\ module may have some
> > > > specific
> > > > > > person in charge.
> > > > > >
> > > > > > No that is not the Apache Way. No one should be in charge of an
> > area.
> > > > > >
> > > > > > Thanks,
> > > > > > Justin
> > > > >
> > > >
> > >
> >
> >
> > --
> > —————————————————
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
>


-- 
—————————————————
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

Re: [DISCUSS] Add the components definition in JIRA template

Posted by jincheng sun <su...@gmail.com>.

Sorry for the late reply, I just came back from team outing.

Thanks for adding more detail information. I have two questions then:

- Is WAL a write strategy for TsFlie, can WAL be incorporated into other
components and what is the reason for being a separate component?
- Can you give an example of what changes belongs to the Other component?

Best,
Jincheng


Jialin Qiao <qi...@apache.org> 于2020年1月9日周四 下午6:39写道：

> Hi,
>
> Sure, maybe a bit more...
>
>   - TsFile : A columnar file format that has flexible schema and data
> management for time series.
>   - Storage : The storage engine of IoTDB, including writing to memtable
> (buffer), flushing, system recovery, and memtable management.
>   - Query : The query process in IoTDB, including multiple types of
> queries, such as raw data query, aggregations, group by time, group by
> device.
>   - Merge : Compaction of data files, including multiple compaction
> strategies, such as (1) merging unsequence TsFile with sequence TsFile, (2)
> merging small sequence TsFiles into a large TsFile.
>   - Sync: Synchronize data files from one IoTDB instance to another.
>   - Authority: Administration of IoTDB, including users, roles and
> privileges.
>   - WAL : Write-ahead-log.
>   - Client: Rpc between client and server, a command-line interface and
> client of different languages.
>   - Distribution : The shared-nothing distribution of IoTDB, including raft
> protocol, data partition, and system information management.
>   - Grafana : The connector for using IoTDB as a data source in Grafana.
>   - Spark-TsFile : A connector that allows users read/write TsFile through
> SparkSQL (treat TsFile as an external data source for SparkSQL).
>   - Spark-IoTDB : A connector that allows users read/write IoTDB through
> SparkSQL (treat IoTDB as an external data source for SparkSQL).
>   - Hive-tsfile : A connector that maps TsFile as a table in Hive for the
> query.
>   - Hadoop-tsfile : A connector that allows users read/write TsFile through
> MapReduce.
>   - Tools : Other tools for IoTDB, such as watermark, memory estimation,
> JMX, query history visualization.
>   - Example : Examples of different modules, such as JDBC, Session, TsFile.
>   - WebSite : IoTDB official website.
>   - Doc: documents of IoTDB, docs folder under root directory.
>   - Other: Those do not belong to the above components.
>
>
> jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午5:28写道：
>
> > Hi,
> >
> > @Xiangdong thank you for helping to clarify my original intention. you
> are
> > right, I really want to list some components to let users choose which
> > component the given issue belongs to.
> >
> > @Jialin I think approach 4 would be more meaningful for IoTDB. Could you
> > please add the detailed description for each component, such as, describe
> > the relationship between `tsfile` and
> > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which changes
> belong
> > to `Other` component, etc., which can help us better understand the
> reasons
> > for such classification. On the other hand, we will further discuss
> whether
> > there is optimization space for this component classification.
> >
> > What do you think?
> >
> > Best,
> > Jincheng
> >
> > Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
> >
> > > >  No that is not the Apache Way. No one should be in charge of an area
> > >
> > > I think Jincheng's mean is, we can list some components to let users
> > > choose which component the given issue belongs to.
> > >
> > > As for which components should be listed, we need to have a discussion.
> > >
> > > > No one should be in charge of an area
> > >
> > > Yes everyone can assign an issue to himself/herself.
> > >
> > > Best,
> > > ----------------------------------
> > > Xiangdong Huang
> > > School of Software, Tsinghua University
> > >
> > >  黄向东
> > > 清华大学 软件学院
> > >
> > >
> > > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午4:38写道：
> > >
> > > > > No that is not the Apache Way. No one should be in charge of an
> area.
> > > >
> > > > Oh, great thanks for correcting me Justin, maybe it's my language
> > > > expression problems. I mean some contributors and committers will be
> > very
> > > > interested in some components, then spend more time, and be more
> > familiar
> > > > with the code.
> > > >
> > > > Best,
> > > > Jincheng
> > > >
> > > >
> > > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 下午4:27写道：
> > > >
> > > > > Hi,
> > > > >
> > > > > > When our community grows stronger, each\ module may have some
> > > specific
> > > > > person in charge.
> > > > >
> > > > > No that is not the Apache Way. No one should be in charge of an
> area.
> > > > >
> > > > > Thanks,
> > > > > Justin
> > > >
> > >
> >
>
>
> --
> —————————————————
> Jialin Qiao
> School of Software, Tsinghua University
>
> 乔嘉林
> 清华大学 软件学院
>

Re: [DISCUSS] Add the components definition in JIRA template

Posted by Jialin Qiao <qi...@apache.org>.

Hi,

Sure, maybe a bit more...

  - TsFile : A columnar file format that has flexible schema and data
management for time series.
  - Storage : The storage engine of IoTDB, including writing to memtable
(buffer), flushing, system recovery, and memtable management.
  - Query : The query process in IoTDB, including multiple types of
queries, such as raw data query, aggregations, group by time, group by
device.
  - Merge : Compaction of data files, including multiple compaction
strategies, such as (1) merging unsequence TsFile with sequence TsFile, (2)
merging small sequence TsFiles into a large TsFile.
  - Sync: Synchronize data files from one IoTDB instance to another.
  - Authority: Administration of IoTDB, including users, roles and
privileges.
  - WAL : Write-ahead-log.
  - Client: Rpc between client and server, a command-line interface and
client of different languages.
  - Distribution : The shared-nothing distribution of IoTDB, including raft
protocol, data partition, and system information management.
  - Grafana : The connector for using IoTDB as a data source in Grafana.
  - Spark-TsFile : A connector that allows users read/write TsFile through
SparkSQL (treat TsFile as an external data source for SparkSQL).
  - Spark-IoTDB : A connector that allows users read/write IoTDB through
SparkSQL (treat IoTDB as an external data source for SparkSQL).
  - Hive-tsfile : A connector that maps TsFile as a table in Hive for the
query.
  - Hadoop-tsfile : A connector that allows users read/write TsFile through
MapReduce.
  - Tools : Other tools for IoTDB, such as watermark, memory estimation,
JMX, query history visualization.
  - Example : Examples of different modules, such as JDBC, Session, TsFile.
  - WebSite : IoTDB official website.
  - Doc: documents of IoTDB, docs folder under root directory.
  - Other: Those do not belong to the above components.


jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午5:28写道：

> Hi,
>
> @Xiangdong thank you for helping to clarify my original intention. you are
> right, I really want to list some components to let users choose which
> component the given issue belongs to.
>
> @Jialin I think approach 4 would be more meaningful for IoTDB. Could you
> please add the detailed description for each component, such as, describe
> the relationship between `tsfile` and
> (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which changes belong
> to `Other` component, etc., which can help us better understand the reasons
> for such classification. On the other hand, we will further discuss whether
> there is optimization space for this component classification.
>
> What do you think?
>
> Best,
> Jincheng
>
> Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：
>
> > >  No that is not the Apache Way. No one should be in charge of an area
> >
> > I think Jincheng's mean is, we can list some components to let users
> > choose which component the given issue belongs to.
> >
> > As for which components should be listed, we need to have a discussion.
> >
> > > No one should be in charge of an area
> >
> > Yes everyone can assign an issue to himself/herself.
> >
> > Best,
> > ----------------------------------
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
> >
> > jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午4:38写道：
> >
> > > > No that is not the Apache Way. No one should be in charge of an area.
> > >
> > > Oh, great thanks for correcting me Justin, maybe it's my language
> > > expression problems. I mean some contributors and committers will be
> very
> > > interested in some components, then spend more time, and be more
> familiar
> > > with the code.
> > >
> > > Best,
> > > Jincheng
> > >
> > >
> > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 下午4:27写道：
> > >
> > > > Hi,
> > > >
> > > > > When our community grows stronger, each\ module may have some
> > specific
> > > > person in charge.
> > > >
> > > > No that is not the Apache Way. No one should be in charge of an area.
> > > >
> > > > Thanks,
> > > > Justin
> > >
> >
>


-- 
—————————————————
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

Re: [DISCUSS] Add the components definition in JIRA template

Posted by jincheng sun <su...@gmail.com>.

Hi,

@Xiangdong thank you for helping to clarify my original intention. you are
right, I really want to list some components to let users choose which
component the given issue belongs to.

@Jialin I think approach 4 would be more meaningful for IoTDB. Could you
please add the detailed description for each component, such as, describe
the relationship between `tsfile` and
(`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which changes belong
to `Other` component, etc., which can help us better understand the reasons
for such classification. On the other hand, we will further discuss whether
there is optimization space for this component classification.

What do you think?

Best,
Jincheng

Xiangdong Huang <sa...@gmail.com> 于2020年1月9日周四 下午4:46写道：

> >  No that is not the Apache Way. No one should be in charge of an area
>
> I think Jincheng's mean is, we can list some components to let users
> choose which component the given issue belongs to.
>
> As for which components should be listed, we need to have a discussion.
>
> > No one should be in charge of an area
>
> Yes everyone can assign an issue to himself/herself.
>
> Best,
> ----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午4:38写道：
>
> > > No that is not the Apache Way. No one should be in charge of an area.
> >
> > Oh, great thanks for correcting me Justin, maybe it's my language
> > expression problems. I mean some contributors and committers will be very
> > interested in some components, then spend more time, and be more familiar
> > with the code.
> >
> > Best,
> > Jincheng
> >
> >
> > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 下午4:27写道：
> >
> > > Hi,
> > >
> > > > When our community grows stronger, each\ module may have some
> specific
> > > person in charge.
> > >
> > > No that is not the Apache Way. No one should be in charge of an area.
> > >
> > > Thanks,
> > > Justin
> >
>

Re: [DISCUSS] Add the components definition in JIRA template

Posted by Xiangdong Huang <sa...@gmail.com>.

>  No that is not the Apache Way. No one should be in charge of an area

I think Jincheng's mean is, we can list some components to let users
choose which component the given issue belongs to.

As for which components should be listed, we need to have a discussion.

> No one should be in charge of an area

Yes everyone can assign an issue to himself/herself.

Best,
----------------------------------
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


jincheng sun <su...@gmail.com> 于2020年1月9日周四 下午4:38写道：

> > No that is not the Apache Way. No one should be in charge of an area.
>
> Oh, great thanks for correcting me Justin, maybe it's my language
> expression problems. I mean some contributors and committers will be very
> interested in some components, then spend more time, and be more familiar
> with the code.
>
> Best,
> Jincheng
>
>
> Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 下午4:27写道：
>
> > Hi,
> >
> > > When our community grows stronger, each\ module may have some specific
> > person in charge.
> >
> > No that is not the Apache Way. No one should be in charge of an area.
> >
> > Thanks,
> > Justin
>

Re: [DISCUSS] Add the components definition in JIRA template

Posted by jincheng sun <su...@gmail.com>.

> No that is not the Apache Way. No one should be in charge of an area.

Oh, great thanks for correcting me Justin, maybe it's my language
expression problems. I mean some contributors and committers will be very
interested in some components, then spend more time, and be more familiar
with the code.

Best,
Jincheng


Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 下午4:27写道：

> Hi,
>
> > When our community grows stronger, each\ module may have some specific
> person in charge.
>
> No that is not the Apache Way. No one should be in charge of an area.
>
> Thanks,
> Justin

Re: [DISCUSS] Add the components definition in JIRA template

Posted by Justin Mclean <ju...@classsoftware.com>.

Hi,

> When our community grows stronger, each\ module may have some specific person in charge.

No that is not the Apache Way. No one should be in charge of an area.

Thanks,
Justin