You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@incubator.apache.org by BLAST <12...@qq.com.INVALID> on 2022/05/26 03:41:07 UTC
回复： [DISCUSSION] Incubating Proposal of Uniffle

+1, Good Luck!




------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "general"                                                                                    <wander4096@gmail.com&gt;;
发送时间:&nbsp;2022年5月25日(星期三) 下午2:26
收件人:&nbsp;"Incubator"<general@incubator.apache.org&gt;;

主题:&nbsp;Re: [DISCUSSION] Incubating Proposal of Uniffle



+1 (binding)

An interesting project. Good luck!

Best,
tison.


Jungtaek Lim <kabhwan.opensource@gmail.com&gt; 于2022年5月25日周三 14:22写道：

&gt; +1 (non-binding)
&gt;
&gt; Good luck!
&gt;
&gt; On Wed, May 25, 2022 at 2:42 PM Daniel Widdis <widdis@gmail.com&gt; wrote:
&gt;
&gt; &gt; This was stated in the other thread: Unified/Universal Shuffle
&gt; &gt;
&gt; &gt; On 5/24/22, 10:04 PM, "XiaoYu" <xiaoyu@apache.org&gt; wrote:
&gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; Hi
&gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; Uniffle&nbsp; as a project name， What does he mean~
&gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; thanks
&gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; Weiwei Yang <wwei@apache.org&gt; 于2022年5月25日周三 12:57写道：
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; +1 (binding)
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; Good luck!
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; On Tue, May 24, 2022 at 8:49 PM Ye Xianjin <advancedxy@gmail.com&gt;
&gt; &gt; wrote:
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; +1 (non-binding).
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; Sent from my iPhone
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt; On May 25, 2022, at 9:59 AM, Goson zhang <
&gt; gosonzhang@apache.org&gt;
&gt; &gt; wrote:
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt; +1 (non-binding)
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt; Good luck!
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt; Daniel Widdis <widdis@gmail.com&gt; 于2022年5月25日周三 09:53写道：
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; +1 (non-binding) from me!&nbsp; Good luck!
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; On 5/24/22, 9:05 AM, "Jerry Shao" <jshao@apache.org&gt; wrote:
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; Hi all,
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; Due to the name issue in thread (
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt; https://lists.apache.org/thread/y07xjkqzvpchncym9zr1hgm3c4l4ql0f),
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; we
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; figured out a new project name "Uniffle" and created a new
&gt; &gt; Thread.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; Please
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; help to discuss.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; We would like to propose Uniffle[1] as a new Apache
&gt; incubator
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; project,
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; you
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; can find the proposal here [2] for more details.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; Uniffle is a high performance, general purpose Remote
&gt; &gt; Shuffle Service
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; for
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; distributed compute engines like Apache Spark
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; <https://spark.apache.org/&gt;, Apache
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; Hadoop MapReduce <https://hadoop.apache.org/&gt;, Apache
&gt; Flink
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; <https://flink.apache.org/&gt; and so on. We are aiming to
&gt; make
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; Firestorm a
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; universal shuffle service for distributed compute engines.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; Shuffle is the key part for a distributed compute engine to
&gt; &gt; exchange
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; the
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; data between distributed tasks, the performance and
&gt; &gt; stability of
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; shuffle
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; will directly affect the whole job. Current “local file
&gt; &gt; pull-like
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; shuffle
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; style” has several limitations:
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1. Current shuffle is hard to support super large
&gt; &gt; workloads,
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; especially
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; in a high load environment, the major problem is IO
&gt; &gt; problem
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; (random
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; disk IO
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; issue, network congestion and timeout).
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2. Current shuffle is hard to deploy on the
&gt; disaggregated
&gt; &gt; compute
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; storage environment, as disk capacity is quite limited
&gt; on
&gt; &gt; compute
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; nodes.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3. The constraint of storing shuffle data locally makes
&gt; &gt; it hard to
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; scale
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; elastically.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; Remote Shuffle Service is the key technology for
&gt; enterprises
&gt; &gt; to build
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; big
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; data platforms, to expand big data applications to
&gt; &gt; disaggregated,
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; online-offline hybrid environments, and to solve above
&gt; &gt; problems.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; The implementation of Remote Shuffle Service -&nbsp; “Uniffle”
&gt; -
&gt; &gt; is
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; heavily
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; adopted in Tencent, and shows its advantages in production.
&gt; &gt; Other
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; enterprises also adopted or prepared to adopt Firestorm in
&gt; &gt; their
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; environments.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; Uniffle's key idea is brought from Salfish shuffle
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; <
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt;
&gt; &gt;
&gt; https://www.researchgate.net/publication/262241541_Sailfish_a_framework_for_large_scale_data_processing
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&gt; ,
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; it has several key design goals:
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1. High performance. Firestorm’s performance is close
&gt; &gt; enough to
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; local
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; file based shuffle style for small workloads. For large
&gt; &gt; workloads,
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; it is
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; far better than the current shuffle style.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2. Fault tolerance. Firestorm provides high availability
&gt; &gt; for
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; Coordinated
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; nodes, and failover for Shuffle nodes.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3. Pluggable. Firestorm is highly pluggable, which could
&gt; &gt; be suited
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; to
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; different compute engines, different backend storages,
&gt; and
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; different
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; wire-protocols.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; We believe that Uniffle project will provide the great
&gt; value
&gt; &gt; for the
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; community if it is accepted by the Apache incubator.
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; I will help this project as champion and many thanks to
&gt; the 3
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; mentors:
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Felix Cheung (felixcheung@apache.org)
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Junping du (junping_du@apache.org)
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Weiwei Yang (wwei@apache.org)
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Xun liu (liuxun@apache.org)
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Zhankun Tang (ztang@apache.org)
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; [1] https://github.com/Tencent/Firestorm
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; [2]
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt; https://cwiki.apache.org/confluence/display/INCUBATOR/UniffleProposal
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; Best regards,
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;&nbsp;&nbsp;&nbsp; Jerry
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt; ---------------------------------------------------------------------
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; To unsubscribe, e-mail:
&gt; &gt; general-unsubscribe@incubator.apache.org
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt; For additional commands, e-mail:
&gt; &gt; general-help@incubator.apache.org
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; &gt;&gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt;
&gt; &gt; ---------------------------------------------------------------------
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt; For additional commands, e-mail:
&gt; general-help@incubator.apache.org
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; &gt; &gt;
&gt; &gt;
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; ---------------------------------------------------------------------
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp; For additional commands, e-mail: general-help@incubator.apache.org
&gt; &gt;
&gt; &gt;
&gt; &gt;
&gt; &gt;
&gt; &gt; ---------------------------------------------------------------------
&gt; &gt; To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
&gt; &gt; For additional commands, e-mail: general-help@incubator.apache.org
&gt; &gt;
&gt; &gt;
&gt;