You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafodion.apache.org by "Ma, Sheng-Chen (Aven)" <sh...@esgyn.cn> on 2016/06/15 09:13:56 UTC
discuss on support hbase 1.1
Hi all:
Apologize for this taking me so long... and thanks Amanda's reply (your reply gives me confidence :)
Last time I had an inspiration to solve support hbase 1.1 without use install_features, but the truth is it's a misconception...
I'd like to summary it first, according to the manual (see https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall ):
1. download source code.
2. source sqenvr.sh & make package-all
3. install Hadoop & hbase
4. install trafodion
Base on the above steps, there is no idea knows which hbase version the user will install. I can't mandatory provisions install Hadoop & hbase before make trafodion package...
So the only way is to predefine hbase version in a config file, so there appear the install_features.
Another question is why not support both hbase 1.0 & 1.1 at the same time?
The answer is hbase1.0 &1.1 have different api , so I can only support one version at one package.
Further question is why not package two package eg: hbase-trx-apache1.0.jar & hbase-trx-apache1.1.jar
It seems a solution, but I don't think it's a good way, reason is similar why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
So I think this release the apache hbase ,we support 1.0...the next release we support 1.1
====================================================================
All above is my view, maybe not correct... if some one good at this pls correct.
Thanks.
-----邮件原件-----
发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
发送时间: 2016年6月4日 2:02
收件人: dev <de...@trafodion.incubator.apache.org>
抄送: Steve Varnau <st...@esgyn.com>
主题: Re: 答复: about pull 503
Hi there Aven-
You can you install_features, actually you must so that the installer knows what version of HBase is supported in that particular package. The installer can install all versions of trafodion. So there will need to be HBase 1.0 support in the installer and HBase 1.1 . install_features is used during installation not during sqenvcom.sh. Take a look at traf_config_check to see the details.
Thanks.
On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <shengchen.ma@esgyn.cn
> wrote:
> hi Steve:
> I think u are right . I will do some change which will not rely on
> install_feature. Then I will have this discussion on dev list.
>
> Thanks
> Aven
>
>
> 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> 发送时间: 2016年6月3日 6:40
> 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> 抄送: Amanda Moran <am...@esgyn.com>
> 主题: RE: about pull 503
>
> Hi Aven,
>
> I think it would be really good to have this discussion on the dev
> list or on github. Others may also be confused by my remarks or may
> have different opinions than I do. Maybe I’m wrong. Either way, it
> would be good record for next person trying to add support of another
> HBase version. But I did not want to post your message without permission, so I’ll reply directly.
>
> Your point #1 about installer script is a good one. But I‘d say that
> the installer script could be enhanced to recognize which version of
> HBase is there and then move only correct TRX file. Either as part of
> your change or separately.
>
> If you are changing build to support apache hbase1.1, then why are you
> trying to retain hbase1.0? If we want both supported, we should be
> able to choose at install time not at build time.
>
> It is hbase-trx makefile that determines which TRX files are built. If
> developr wants to switch hbase version, they should change makefile,
> and maybe the install_features file too. But the sqenvcom.sh file
> should not read the install_features file. Your changes work
> differently than all the other versions of TRX. I would prefer consistency.
>
> --Steve
>
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> shengchen.ma@esgyn.cn>]
> Sent: Thursday, June 2, 2016 3:02 AM
> To: Steve Varnau
> <st...@esgyn.com>>
> Subject: about pull 503
>
> Hi Steve:
>
> I send this mail to discuss #503 (
> https://github.com/apache/incubator-trafodion/pull/503)
> As your comment is a little confused me, I’d like to make sure something.
>
> As u commented:
>
> In other cases, we support multiple versions simultaneously by
> generating multiple TRX files. I'm not sure we want to add more TRX
> files, but I'm also not sure we want to switch how the build works
> based on content of install_features file.
>
> Previously it works the other way round -- the install_features file
> indicates what TRX files are available, hence which distros are supported.
>
> This change will not add new trx file ,it just change
> hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> apache-trx.jar, because there are two reason, one is our install
> script will add hbase-trx-apache*.jar to classpath, see
> /incubator-trafodion/install/installer/traf_apache_mods
> line51. Two is if 2 different version hbase-trx-apache*.jar added to
> classpath, hbase will only use the first meet class in the jar
> package, it may lead to conflict.
>
> About install_features file. I am not sure we want to switch how the
> build works based on content of install_features file either. But the
> only way I can distinguish which hbase version to use is this file. If
> there is another place to distinguish this ,pls tell me ,thanks.
>
> About your last sentence , I am really confuse, is that means we
> indeed use install_features to distinguish hbase version?
>
>
> Thanks,
> Aven
>
>
--
Thanks,
Amanda Moran
RE: discuss on support hbase 1.1
Posted by "Jin, Jian (Seth)" <Ji...@esgyn.cn>.
I prefer to stay with Selva's opinion. If we have large matrix to support different Hadoop and Hbase version in ONE Trafodion release, the testing effort is also huge.
Br,
Seth
-----Original Message-----
From: Qifan Chen [mailto:qifan.chen@esgyn.com]
Sent: 2016年6月22日 1:25
To: dev <de...@trafodion.incubator.apache.org>
Subject: Re: discuss on support hbase 1.1
I personally would prefer that a common Trafodion code base to deal with different storage engines (HBase, Hive etc), in different versions. This will be a ideal situation for feature development and code maintenance.
The initial requirement is that only one version of a storage engine per sqstart instance.
Within SQL, it probably is a straight forward task to call
javaObjectInterface::buildClassPath() method to load JVM with a specific class path to a specific JAR file for that version of the storage engine.
1. We can set CLASSPATH during install to point to
/home/trafodion/0621a/hbase1.0/jars if asked to run against Hbase 1.0.
2. we can set CLASSPATH during install to point to
/home/trafodion/0621a/hbase1.1/jars if asked to run against Hbase 1.1.
Of course, within SQL, we need to distinguish calls to different versions of storage engines. Maybe hide all these under a layer of abstraction.
--Qifan
int JavaObjectInterface::createJVM()
{
JavaVMInitArgs jvm_args;
JavaVMOption jvm_options[MAX_NO_JVM_OPTIONS];
char* classPathArg = buildClassPath();
int numJVMOptions = 0;
jvm_options[numJVMOptions].optionString = classPathArg;
QRLogger::log(CAT_SQL_HDFS_JNI_TOP, LL_DEBUG, "Using classpath: %s",
jvm_options[numJVMOptions].optionString);
numJVMOptions++;
char maxHeapOptions[64];
bool passMaxHeapToJVM = true;
int maxHeapEnvvarMB = DEFAULT_JVM_MAX_HEAP_SIZE;
const char *maxHeapSizeStr = getenv("JVM_MAX_HEAP_SIZE_MB");
if (maxHeapSizeStr)
{
maxHeapEnvvarMB = atoi(maxHeapSizeStr);
if (maxHeapEnvvarMB <= 0)
passMaxHeapToJVM = false;
}
char* JavaObjectInterface::buildClassPath()
{
char* classPath = getenv("CLASSPATH");
Int32 size = strlen(classPath) + 128;
char* classPathBuffer = (char*)malloc(size);
strcpy(classPathBuffer, "-Djava.class.path=");
if (isHBaseCompatibilityMode())
strcat(classPathBuffer,
"/opt/home/tools/hbase-0.94.5/lib/hadoop-core-1.0.4.jar:");
strcat(classPathBuffer, classPath);
return classPathBuffer;
}
On Tue, Jun 21, 2016 at 11:36 AM, Amanda Moran <am...@esgyn.com>
wrote:
> So, it narrows down the question of should Trafodion support both
> Apache Hbase 1.0 and Apache Hbase 1.1 from the same code base. Or is
> it ok to say Trafodion 2.0 version needs Apache hbase 1.0 and
> Trafodion 2.1 version needs Apache hbase 1.1.
>
> +1 to this.
>
> It's always good to support has many versions as possible, but
> sometimes it is just too difficult.
>
> On Tue, Jun 21, 2016 at 8:41 AM, Selva Govindarajan <
> selva.govindarajan@esgyn.com> wrote:
>
> > My understanding is that install_features file lists the Trafodion
> required
> > products and its version. At the time of install, if the dependent
> product
> > listed in this file doesn't exist or the version below exists in the
> > cluster, install will fail.
> >
> > With the change in sqenvcom.sh, you are determining which version of
> > Trx-jar to be built for apache hbase version. This would mean,
> >
> > Change install_features to build Apache_1.0, build trx jar files
> correspond
> > to Apache hbase 1.0. And then change install_features to build
> Apache_1.1,
> > build trx jar files correspond to Apache hbase 1.1.
> >
> > Trafodion build process doesn't support this concept.
> >
> > So, it narrows down the question of should Trafodion support both
> > Apache Hbase 1.0 and Apache Hbase 1.1 from the same code base. Or
> > is it ok to say Trafodion 2.0 version needs Apache hbase 1.0 and
> > Trafodion 2.1 version needs Apache hbase 1.1.
> >
> > My vote is for the latter option.
> >
> > If so, basically you just need to remove the change in sqenvcom.sh
> > and rework it to build for Apache hbase 1.1.
> >
> > Selva
> >
> > -----Original Message-----
> > From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > Sent: Tuesday, June 21, 2016 2:43 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: 答复: discuss on support hbase 1.1
> >
> > Hi
> > Seemingly it's not a good solution to use install_features to
> > determine which trx file will use. I 'm try to add more trx files.
> > So that install_features file will just indicate what trx files are
> > available
> >
> >
> > -----邮件原件-----
> > 发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > 发送时间: 2016年6月21日 13:15
> > 收件人: dev@trafodion.incubator.apache.org
> > 主题: 答复: discuss on support hbase 1.1
> >
> > Quote Steve's view first
> > "In other cases, we support multiple versions simultaneously by
> generating
> > multiple TRX files. I'm not sure we want to add more TRX files, but
> > I'm also not sure we want to switch how the build works based on
> > content of install_features file.
> >
> > Previously it works the other way round -- the install_features file
> > indicates what TRX files are available, hence which distros are
> supported."
> >
> > No more trx files will be added.
> > Have to use install_features to decide whice hbase version will be
> > used.(discussed in the previous msg)
> >
> >
> > -----邮件原件-----
> > 发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
> > 发送时间: 2016年6月21日 12:51
> > 收件人: dev@trafodion.incubator.apache.org
> > 主题: RE: discuss on support hbase 1.1
> >
> > I have the same observation as that of Steve for PR 503. Can you
> > please address them?
> >
> > Selva
> >
> > -----Original Message-----
> > From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > Sent: Monday, June 20, 2016 9:00 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: 答复: discuss on support hbase 1.1
> >
> > Hi All:
> > Can someone review #503
> > (https://github.com/apache/incubator-trafodion/pull/503 )?
> > Since there is a long time ,and several time discussion shows it's
> > the
> only
> > solution.
> >
> > Thanks
> >
> > -----邮件原件-----
> > 发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > 发送时间: 2016年6月16日 10:51
> > 收件人: dev@trafodion.incubator.apache.org
> > 主题: 答复: discuss on support hbase 1.1
> >
> > Yes reflection is a good option, but I have do some research , it's
> > hard
> to
> > use reflection,
> > eg: TrxTransactionState.java need add @Override to next(..) for
> > hbase
> 1.1,
> > and this I think can't use reflect #ifndef HDP2.3 HBASE1.1
> > @Override
> > #endif
> > public synchronized boolean next(final List<Cell> results)
> > throws IOException {
> > return next(results, -1);
> > }
> >
> > Another is extends from new class, ScheduledChore is new class in
> hbase1.1
> > while Chore is disappear in hbase 1.0 #ifdef HDP2.3 HBASE1.1 public
> > class CleanOldTransactionsChore extends ScheduledChore { #else
> > public class CleanOldTransactionsChore extends Chore { #endif
> >
> > And there still some kind of changes I can hardly use reflection...
> >
> >
> >
> > -----邮件原件-----
> > 发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
> > 发送时间: 2016年6月15日 22:36
> > 收件人: dev@trafodion.incubator.apache.org
> > 主题: RE: discuss on support hbase 1.1
> >
> > Another option is to check the code where Trafodion issues different
> > APIs and rework so that common set of API can be used. Reflection
> > concept in java might also work in some cases. If the rework is not
> > possible then it
> might
> > be necessary to stick with higher version.
> >
> > Selva
> >
> > -----Original Message-----
> > From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > Sent: Wednesday, June 15, 2016 2:14 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: discuss on support hbase 1.1
> >
> > Hi all:
> > Apologize for this taking me so long... and thanks Amanda's reply
> > (your reply gives me confidence :)
> >
> > Last time I had an inspiration to solve support hbase 1.1 without
> > use install_features, but the truth is it's a misconception...
> >
> > I'd like to summary it first, according to the manual (see
> >
> >
> https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#Bui
> ldSource-makeall
> > )
> > :
> > 1. download source code.
> > 2. source sqenvr.sh & make package-all
> > 3. install Hadoop & hbase
> > 4. install trafodion
> >
> > Base on the above steps, there is no idea knows which hbase version
> > the user will install. I can't mandatory provisions install Hadoop
> > & hbase before make trafodion package...
> > So the only way is to predefine hbase version in a config file, so
> > there appear the install_features.
> >
> > Another question is why not support both hbase 1.0 & 1.1 at the same
> time?
> > The answer is hbase1.0 &1.1 have different api , so I can
> > only support one version at one package.
> > Further question is why not package two package eg:
> hbase-trx-apache1.0.jar
> > & hbase-trx-apache1.1.jar
> > It seems a solution, but I don't think it's a good way,
> > reason is similar why there is only hbase-trx-cdh5_4.jar and
> > hbase-trx-hdp2_3.jar.
> >
> > So I think this release the apache hbase ,we support 1.0...the next
> release
> > we support 1.1
> > ====================================================================
> > All above is my view, maybe not correct... if some one good at this
> > pls correct.
> >
> > Thanks.
> >
> >
> > -----邮件原件-----
> > 发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
> > 发送时间: 2016年6月4日 2:02
> > 收件人: dev <de...@trafodion.incubator.apache.org>
> > 抄送: Steve Varnau <st...@esgyn.com>
> > 主题: Re: 答复: about pull 503
> >
> > Hi there Aven-
> >
> > You can you install_features, actually you must so that the
> > installer
> knows
> > what version of HBase is supported in that particular package. The
> > installer can install all versions of trafodion. So there will need
> > to be HBase 1.0 support in the installer and HBase 1.1 .
> > install_features is used during installation not during sqenvcom.sh.
> > Take a look at traf_config_check to see the details.
> >
> > Thanks.
> >
> > On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <
> > shengchen.ma@esgyn.cn
> > > wrote:
> >
> > > hi Steve:
> > > I think u are right . I will do some change which will not rely on
> > > install_feature. Then I will have this discussion on dev list.
> > >
> > > Thanks
> > > Aven
> > >
> > >
> > > 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> > > 发送时间: 2016年6月3日 6:40
> > > 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> > > 抄送: Amanda Moran <am...@esgyn.com>
> > > 主题: RE: about pull 503
> > >
> > > Hi Aven,
> > >
> > > I think it would be really good to have this discussion on the dev
> > > list or on github. Others may also be confused by my remarks or
> > > may have different opinions than I do. Maybe I’m wrong. Either
> > > way, it would be good record for next person trying to add support
> > > of another HBase version. But I did not want to post your message
> > > without permission, so I’ll reply directly.
> > >
> > > Your point #1 about installer script is a good one. But I‘d say
> > > that the installer script could be enhanced to recognize which
> > > version of HBase is there and then move only correct TRX file.
> > > Either as part of your change or separately.
> > >
> > > If you are changing build to support apache hbase1.1, then why are
> > > you trying to retain hbase1.0? If we want both supported, we
> > > should be able to choose at install time not at build time.
> > >
> > > It is hbase-trx makefile that determines which TRX files are
> > > built. If developr wants to switch hbase version, they should
> > > change makefile, and maybe the install_features file too. But the
> > > sqenvcom.sh file should not read the install_features file. Your
> > > changes work differently than all the other versions of TRX. I
> > > would prefer consistency.
> > >
> > > --Steve
> > >
> > > From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> > > shengchen.ma@esgyn.cn>]
> > > Sent: Thursday, June 2, 2016 3:02 AM
> > > To: Steve Varnau
> > > <st...@esgyn.com>>
> > > Subject: about pull 503
> > >
> > > Hi Steve:
> > >
> > > I send this mail to discuss #503 (
> > > https://github.com/apache/incubator-trafodion/pull/503)
> > > As your comment is a little confused me, I’d like to make sure
> something.
> > >
> > > As u commented:
> > >
> > > In other cases, we support multiple versions simultaneously by
> > > generating multiple TRX files. I'm not sure we want to add more
> > > TRX files, but I'm also not sure we want to switch how the build
> > > works based on content of install_features file.
> > >
> > > Previously it works the other way round -- the install_features
> > > file indicates what TRX files are available, hence which distros
> > > are
> > supported.
> > >
> > > This change will not add new trx file ,it just change
> > > hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> > > apache-trx.jar, because there are two reason, one is our install
> > > script will add hbase-trx-apache*.jar to classpath, see
> > > /incubator-trafodion/install/installer/traf_apache_mods
> > > line51. Two is if 2 different version hbase-trx-apache*.jar added
> > > to classpath, hbase will only use the first meet class in the jar
> > > package, it may lead to conflict.
> > >
> > > About install_features file. I am not sure we want to switch how
> > > the build works based on content of install_features file either.
> > > But the only way I can distinguish which hbase version to use is
> > > this file. If there is another place to distinguish this ,pls tell me ,thanks.
> > >
> > > About your last sentence , I am really confuse, is that means we
> > > indeed use install_features to distinguish hbase version?
> > >
> > >
> > > Thanks,
> > > Aven
> > >
> > >
> >
> >
> > --
> > Thanks,
> >
> > Amanda Moran
> >
>
>
>
> --
> Thanks,
>
> Amanda Moran
>
--
Regards, --Qifan
Re: discuss on support hbase 1.1
Posted by Qifan Chen <qi...@esgyn.com>.
I personally would prefer that a common Trafodion code base to deal with
different storage engines (HBase, Hive etc), in different versions. This
will be a ideal situation for feature development and code maintenance.
The initial requirement is that only one version of a storage engine per
sqstart instance.
Within SQL, it probably is a straight forward task to call
javaObjectInterface::buildClassPath() method to load JVM with a specific
class path to a specific JAR file for that version of the storage engine.
1. We can set CLASSPATH during install to point to
/home/trafodion/0621a/hbase1.0/jars if asked to run against Hbase 1.0.
2. we can set CLASSPATH during install to point to
/home/trafodion/0621a/hbase1.1/jars if asked to run against Hbase 1.1.
Of course, within SQL, we need to distinguish calls to different versions
of storage engines. Maybe hide all these under a layer of abstraction.
--Qifan
int JavaObjectInterface::createJVM()
{
JavaVMInitArgs jvm_args;
JavaVMOption jvm_options[MAX_NO_JVM_OPTIONS];
char* classPathArg = buildClassPath();
int numJVMOptions = 0;
jvm_options[numJVMOptions].optionString = classPathArg;
QRLogger::log(CAT_SQL_HDFS_JNI_TOP, LL_DEBUG, "Using classpath: %s",
jvm_options[numJVMOptions].optionString);
numJVMOptions++;
char maxHeapOptions[64];
bool passMaxHeapToJVM = true;
int maxHeapEnvvarMB = DEFAULT_JVM_MAX_HEAP_SIZE;
const char *maxHeapSizeStr = getenv("JVM_MAX_HEAP_SIZE_MB");
if (maxHeapSizeStr)
{
maxHeapEnvvarMB = atoi(maxHeapSizeStr);
if (maxHeapEnvvarMB <= 0)
passMaxHeapToJVM = false;
}
char* JavaObjectInterface::buildClassPath()
{
char* classPath = getenv("CLASSPATH");
Int32 size = strlen(classPath) + 128;
char* classPathBuffer = (char*)malloc(size);
strcpy(classPathBuffer, "-Djava.class.path=");
if (isHBaseCompatibilityMode())
strcat(classPathBuffer,
"/opt/home/tools/hbase-0.94.5/lib/hadoop-core-1.0.4.jar:");
strcat(classPathBuffer, classPath);
return classPathBuffer;
}
On Tue, Jun 21, 2016 at 11:36 AM, Amanda Moran <am...@esgyn.com>
wrote:
> So, it narrows down the question of should Trafodion support both Apache
> Hbase 1.0 and Apache Hbase 1.1 from the same code base. Or is it ok to
> say
> Trafodion 2.0 version needs Apache hbase 1.0 and Trafodion 2.1 version
> needs
> Apache hbase 1.1.
>
> +1 to this.
>
> It's always good to support has many versions as possible, but sometimes it
> is just too difficult.
>
> On Tue, Jun 21, 2016 at 8:41 AM, Selva Govindarajan <
> selva.govindarajan@esgyn.com> wrote:
>
> > My understanding is that install_features file lists the Trafodion
> required
> > products and its version. At the time of install, if the dependent
> product
> > listed in this file doesn't exist or the version below exists in the
> > cluster, install will fail.
> >
> > With the change in sqenvcom.sh, you are determining which version of
> > Trx-jar
> > to be built for apache hbase version. This would mean,
> >
> > Change install_features to build Apache_1.0, build trx jar files
> correspond
> > to Apache hbase 1.0. And then change install_features to build
> Apache_1.1,
> > build trx jar files correspond to Apache hbase 1.1.
> >
> > Trafodion build process doesn't support this concept.
> >
> > So, it narrows down the question of should Trafodion support both Apache
> > Hbase 1.0 and Apache Hbase 1.1 from the same code base. Or is it ok to
> > say
> > Trafodion 2.0 version needs Apache hbase 1.0 and Trafodion 2.1 version
> > needs
> > Apache hbase 1.1.
> >
> > My vote is for the latter option.
> >
> > If so, basically you just need to remove the change in sqenvcom.sh and
> > rework it to build for Apache hbase 1.1.
> >
> > Selva
> >
> > -----Original Message-----
> > From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > Sent: Tuesday, June 21, 2016 2:43 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: 答复: discuss on support hbase 1.1
> >
> > Hi
> > Seemingly it's not a good solution to use install_features to determine
> > which trx file will use. I 'm try to add more trx files. So that
> > install_features file will just indicate what trx files are available
> >
> >
> > -----邮件原件-----
> > 发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > 发送时间: 2016年6月21日 13:15
> > 收件人: dev@trafodion.incubator.apache.org
> > 主题: 答复: discuss on support hbase 1.1
> >
> > Quote Steve's view first
> > "In other cases, we support multiple versions simultaneously by
> generating
> > multiple TRX files. I'm not sure we want to add more TRX files, but I'm
> > also
> > not sure we want to switch how the build works based on content of
> > install_features file.
> >
> > Previously it works the other way round -- the install_features file
> > indicates what TRX files are available, hence which distros are
> supported."
> >
> > No more trx files will be added.
> > Have to use install_features to decide whice hbase version will be
> > used.(discussed in the previous msg)
> >
> >
> > -----邮件原件-----
> > 发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
> > 发送时间: 2016年6月21日 12:51
> > 收件人: dev@trafodion.incubator.apache.org
> > 主题: RE: discuss on support hbase 1.1
> >
> > I have the same observation as that of Steve for PR 503. Can you please
> > address them?
> >
> > Selva
> >
> > -----Original Message-----
> > From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > Sent: Monday, June 20, 2016 9:00 PM
> > To: dev@trafodion.incubator.apache.org
> > Subject: 答复: discuss on support hbase 1.1
> >
> > Hi All:
> > Can someone review #503
> > (https://github.com/apache/incubator-trafodion/pull/503 )?
> > Since there is a long time ,and several time discussion shows it's the
> only
> > solution.
> >
> > Thanks
> >
> > -----邮件原件-----
> > 发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > 发送时间: 2016年6月16日 10:51
> > 收件人: dev@trafodion.incubator.apache.org
> > 主题: 答复: discuss on support hbase 1.1
> >
> > Yes reflection is a good option, but I have do some research , it's hard
> to
> > use reflection,
> > eg: TrxTransactionState.java need add @Override to next(..) for hbase
> 1.1,
> > and this I think can't use reflect #ifndef HDP2.3 HBASE1.1
> > @Override
> > #endif
> > public synchronized boolean next(final List<Cell> results) throws
> > IOException {
> > return next(results, -1);
> > }
> >
> > Another is extends from new class, ScheduledChore is new class in
> hbase1.1
> > while Chore is disappear in hbase 1.0 #ifdef HDP2.3 HBASE1.1 public class
> > CleanOldTransactionsChore extends ScheduledChore { #else public class
> > CleanOldTransactionsChore extends Chore { #endif
> >
> > And there still some kind of changes I can hardly use reflection...
> >
> >
> >
> > -----邮件原件-----
> > 发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
> > 发送时间: 2016年6月15日 22:36
> > 收件人: dev@trafodion.incubator.apache.org
> > 主题: RE: discuss on support hbase 1.1
> >
> > Another option is to check the code where Trafodion issues different APIs
> > and rework so that common set of API can be used. Reflection concept in
> > java
> > might also work in some cases. If the rework is not possible then it
> might
> > be necessary to stick with higher version.
> >
> > Selva
> >
> > -----Original Message-----
> > From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> > Sent: Wednesday, June 15, 2016 2:14 AM
> > To: dev@trafodion.incubator.apache.org
> > Subject: discuss on support hbase 1.1
> >
> > Hi all:
> > Apologize for this taking me so long... and thanks Amanda's reply (your
> > reply gives me confidence :)
> >
> > Last time I had an inspiration to solve support hbase 1.1 without use
> > install_features, but the truth is it's a misconception...
> >
> > I'd like to summary it first, according to the manual (see
> >
> >
> https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall
> > )
> > :
> > 1. download source code.
> > 2. source sqenvr.sh & make package-all
> > 3. install Hadoop & hbase
> > 4. install trafodion
> >
> > Base on the above steps, there is no idea knows which hbase version the
> > user
> > will install. I can't mandatory provisions install Hadoop & hbase before
> > make trafodion package...
> > So the only way is to predefine hbase version in a config file, so there
> > appear the install_features.
> >
> > Another question is why not support both hbase 1.0 & 1.1 at the same
> time?
> > The answer is hbase1.0 &1.1 have different api , so I can only
> > support one
> > version at one package.
> > Further question is why not package two package eg:
> hbase-trx-apache1.0.jar
> > & hbase-trx-apache1.1.jar
> > It seems a solution, but I don't think it's a good way, reason is
> > similar
> > why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
> >
> > So I think this release the apache hbase ,we support 1.0...the next
> release
> > we support 1.1
> > ====================================================================
> > All above is my view, maybe not correct... if some one good at this pls
> > correct.
> >
> > Thanks.
> >
> >
> > -----邮件原件-----
> > 发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
> > 发送时间: 2016年6月4日 2:02
> > 收件人: dev <de...@trafodion.incubator.apache.org>
> > 抄送: Steve Varnau <st...@esgyn.com>
> > 主题: Re: 答复: about pull 503
> >
> > Hi there Aven-
> >
> > You can you install_features, actually you must so that the installer
> knows
> > what version of HBase is supported in that particular package. The
> > installer
> > can install all versions of trafodion. So there will need to be HBase 1.0
> > support in the installer and HBase 1.1 . install_features is used during
> > installation not during sqenvcom.sh. Take a look at traf_config_check to
> > see
> > the details.
> >
> > Thanks.
> >
> > On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <
> > shengchen.ma@esgyn.cn
> > > wrote:
> >
> > > hi Steve:
> > > I think u are right . I will do some change which will not rely on
> > > install_feature. Then I will have this discussion on dev list.
> > >
> > > Thanks
> > > Aven
> > >
> > >
> > > 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> > > 发送时间: 2016年6月3日 6:40
> > > 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> > > 抄送: Amanda Moran <am...@esgyn.com>
> > > 主题: RE: about pull 503
> > >
> > > Hi Aven,
> > >
> > > I think it would be really good to have this discussion on the dev
> > > list or on github. Others may also be confused by my remarks or may
> > > have different opinions than I do. Maybe I’m wrong. Either way, it
> > > would be good record for next person trying to add support of another
> > > HBase version. But I did not want to post your message without
> > > permission, so I’ll reply directly.
> > >
> > > Your point #1 about installer script is a good one. But I‘d say that
> > > the installer script could be enhanced to recognize which version of
> > > HBase is there and then move only correct TRX file. Either as part of
> > > your change or separately.
> > >
> > > If you are changing build to support apache hbase1.1, then why are you
> > > trying to retain hbase1.0? If we want both supported, we should be
> > > able to choose at install time not at build time.
> > >
> > > It is hbase-trx makefile that determines which TRX files are built. If
> > > developr wants to switch hbase version, they should change makefile,
> > > and maybe the install_features file too. But the sqenvcom.sh file
> > > should not read the install_features file. Your changes work
> > > differently than all the other versions of TRX. I would prefer
> > > consistency.
> > >
> > > --Steve
> > >
> > > From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> > > shengchen.ma@esgyn.cn>]
> > > Sent: Thursday, June 2, 2016 3:02 AM
> > > To: Steve Varnau
> > > <st...@esgyn.com>>
> > > Subject: about pull 503
> > >
> > > Hi Steve:
> > >
> > > I send this mail to discuss #503 (
> > > https://github.com/apache/incubator-trafodion/pull/503)
> > > As your comment is a little confused me, I’d like to make sure
> something.
> > >
> > > As u commented:
> > >
> > > In other cases, we support multiple versions simultaneously by
> > > generating multiple TRX files. I'm not sure we want to add more TRX
> > > files, but I'm also not sure we want to switch how the build works
> > > based on content of install_features file.
> > >
> > > Previously it works the other way round -- the install_features file
> > > indicates what TRX files are available, hence which distros are
> > supported.
> > >
> > > This change will not add new trx file ,it just change
> > > hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> > > apache-trx.jar, because there are two reason, one is our install
> > > script will add hbase-trx-apache*.jar to classpath, see
> > > /incubator-trafodion/install/installer/traf_apache_mods
> > > line51. Two is if 2 different version hbase-trx-apache*.jar added to
> > > classpath, hbase will only use the first meet class in the jar
> > > package, it may lead to conflict.
> > >
> > > About install_features file. I am not sure we want to switch how the
> > > build works based on content of install_features file either. But the
> > > only way I can distinguish which hbase version to use is this file. If
> > > there is another place to distinguish this ,pls tell me ,thanks.
> > >
> > > About your last sentence , I am really confuse, is that means we
> > > indeed use install_features to distinguish hbase version?
> > >
> > >
> > > Thanks,
> > > Aven
> > >
> > >
> >
> >
> > --
> > Thanks,
> >
> > Amanda Moran
> >
>
>
>
> --
> Thanks,
>
> Amanda Moran
>
--
Regards, --Qifan
Re: discuss on support hbase 1.1
Posted by Amanda Moran <am...@esgyn.com>.
So, it narrows down the question of should Trafodion support both Apache
Hbase 1.0 and Apache Hbase 1.1 from the same code base. Or is it ok to say
Trafodion 2.0 version needs Apache hbase 1.0 and Trafodion 2.1 version needs
Apache hbase 1.1.
+1 to this.
It's always good to support has many versions as possible, but sometimes it
is just too difficult.
On Tue, Jun 21, 2016 at 8:41 AM, Selva Govindarajan <
selva.govindarajan@esgyn.com> wrote:
> My understanding is that install_features file lists the Trafodion required
> products and its version. At the time of install, if the dependent product
> listed in this file doesn't exist or the version below exists in the
> cluster, install will fail.
>
> With the change in sqenvcom.sh, you are determining which version of
> Trx-jar
> to be built for apache hbase version. This would mean,
>
> Change install_features to build Apache_1.0, build trx jar files correspond
> to Apache hbase 1.0. And then change install_features to build Apache_1.1,
> build trx jar files correspond to Apache hbase 1.1.
>
> Trafodion build process doesn't support this concept.
>
> So, it narrows down the question of should Trafodion support both Apache
> Hbase 1.0 and Apache Hbase 1.1 from the same code base. Or is it ok to
> say
> Trafodion 2.0 version needs Apache hbase 1.0 and Trafodion 2.1 version
> needs
> Apache hbase 1.1.
>
> My vote is for the latter option.
>
> If so, basically you just need to remove the change in sqenvcom.sh and
> rework it to build for Apache hbase 1.1.
>
> Selva
>
> -----Original Message-----
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> Sent: Tuesday, June 21, 2016 2:43 AM
> To: dev@trafodion.incubator.apache.org
> Subject: 答复: discuss on support hbase 1.1
>
> Hi
> Seemingly it's not a good solution to use install_features to determine
> which trx file will use. I 'm try to add more trx files. So that
> install_features file will just indicate what trx files are available
>
>
> -----邮件原件-----
> 发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> 发送时间: 2016年6月21日 13:15
> 收件人: dev@trafodion.incubator.apache.org
> 主题: 答复: discuss on support hbase 1.1
>
> Quote Steve's view first
> "In other cases, we support multiple versions simultaneously by generating
> multiple TRX files. I'm not sure we want to add more TRX files, but I'm
> also
> not sure we want to switch how the build works based on content of
> install_features file.
>
> Previously it works the other way round -- the install_features file
> indicates what TRX files are available, hence which distros are supported."
>
> No more trx files will be added.
> Have to use install_features to decide whice hbase version will be
> used.(discussed in the previous msg)
>
>
> -----邮件原件-----
> 发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
> 发送时间: 2016年6月21日 12:51
> 收件人: dev@trafodion.incubator.apache.org
> 主题: RE: discuss on support hbase 1.1
>
> I have the same observation as that of Steve for PR 503. Can you please
> address them?
>
> Selva
>
> -----Original Message-----
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> Sent: Monday, June 20, 2016 9:00 PM
> To: dev@trafodion.incubator.apache.org
> Subject: 答复: discuss on support hbase 1.1
>
> Hi All:
> Can someone review #503
> (https://github.com/apache/incubator-trafodion/pull/503 )?
> Since there is a long time ,and several time discussion shows it's the only
> solution.
>
> Thanks
>
> -----邮件原件-----
> 发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> 发送时间: 2016年6月16日 10:51
> 收件人: dev@trafodion.incubator.apache.org
> 主题: 答复: discuss on support hbase 1.1
>
> Yes reflection is a good option, but I have do some research , it's hard to
> use reflection,
> eg: TrxTransactionState.java need add @Override to next(..) for hbase 1.1,
> and this I think can't use reflect #ifndef HDP2.3 HBASE1.1
> @Override
> #endif
> public synchronized boolean next(final List<Cell> results) throws
> IOException {
> return next(results, -1);
> }
>
> Another is extends from new class, ScheduledChore is new class in hbase1.1
> while Chore is disappear in hbase 1.0 #ifdef HDP2.3 HBASE1.1 public class
> CleanOldTransactionsChore extends ScheduledChore { #else public class
> CleanOldTransactionsChore extends Chore { #endif
>
> And there still some kind of changes I can hardly use reflection...
>
>
>
> -----邮件原件-----
> 发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
> 发送时间: 2016年6月15日 22:36
> 收件人: dev@trafodion.incubator.apache.org
> 主题: RE: discuss on support hbase 1.1
>
> Another option is to check the code where Trafodion issues different APIs
> and rework so that common set of API can be used. Reflection concept in
> java
> might also work in some cases. If the rework is not possible then it might
> be necessary to stick with higher version.
>
> Selva
>
> -----Original Message-----
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
> Sent: Wednesday, June 15, 2016 2:14 AM
> To: dev@trafodion.incubator.apache.org
> Subject: discuss on support hbase 1.1
>
> Hi all:
> Apologize for this taking me so long... and thanks Amanda's reply (your
> reply gives me confidence :)
>
> Last time I had an inspiration to solve support hbase 1.1 without use
> install_features, but the truth is it's a misconception...
>
> I'd like to summary it first, according to the manual (see
>
> https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall
> )
> :
> 1. download source code.
> 2. source sqenvr.sh & make package-all
> 3. install Hadoop & hbase
> 4. install trafodion
>
> Base on the above steps, there is no idea knows which hbase version the
> user
> will install. I can't mandatory provisions install Hadoop & hbase before
> make trafodion package...
> So the only way is to predefine hbase version in a config file, so there
> appear the install_features.
>
> Another question is why not support both hbase 1.0 & 1.1 at the same time?
> The answer is hbase1.0 &1.1 have different api , so I can only
> support one
> version at one package.
> Further question is why not package two package eg: hbase-trx-apache1.0.jar
> & hbase-trx-apache1.1.jar
> It seems a solution, but I don't think it's a good way, reason is
> similar
> why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
>
> So I think this release the apache hbase ,we support 1.0...the next release
> we support 1.1
> ====================================================================
> All above is my view, maybe not correct... if some one good at this pls
> correct.
>
> Thanks.
>
>
> -----邮件原件-----
> 发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
> 发送时间: 2016年6月4日 2:02
> 收件人: dev <de...@trafodion.incubator.apache.org>
> 抄送: Steve Varnau <st...@esgyn.com>
> 主题: Re: 答复: about pull 503
>
> Hi there Aven-
>
> You can you install_features, actually you must so that the installer knows
> what version of HBase is supported in that particular package. The
> installer
> can install all versions of trafodion. So there will need to be HBase 1.0
> support in the installer and HBase 1.1 . install_features is used during
> installation not during sqenvcom.sh. Take a look at traf_config_check to
> see
> the details.
>
> Thanks.
>
> On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <
> shengchen.ma@esgyn.cn
> > wrote:
>
> > hi Steve:
> > I think u are right . I will do some change which will not rely on
> > install_feature. Then I will have this discussion on dev list.
> >
> > Thanks
> > Aven
> >
> >
> > 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> > 发送时间: 2016年6月3日 6:40
> > 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> > 抄送: Amanda Moran <am...@esgyn.com>
> > 主题: RE: about pull 503
> >
> > Hi Aven,
> >
> > I think it would be really good to have this discussion on the dev
> > list or on github. Others may also be confused by my remarks or may
> > have different opinions than I do. Maybe I’m wrong. Either way, it
> > would be good record for next person trying to add support of another
> > HBase version. But I did not want to post your message without
> > permission, so I’ll reply directly.
> >
> > Your point #1 about installer script is a good one. But I‘d say that
> > the installer script could be enhanced to recognize which version of
> > HBase is there and then move only correct TRX file. Either as part of
> > your change or separately.
> >
> > If you are changing build to support apache hbase1.1, then why are you
> > trying to retain hbase1.0? If we want both supported, we should be
> > able to choose at install time not at build time.
> >
> > It is hbase-trx makefile that determines which TRX files are built. If
> > developr wants to switch hbase version, they should change makefile,
> > and maybe the install_features file too. But the sqenvcom.sh file
> > should not read the install_features file. Your changes work
> > differently than all the other versions of TRX. I would prefer
> > consistency.
> >
> > --Steve
> >
> > From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> > shengchen.ma@esgyn.cn>]
> > Sent: Thursday, June 2, 2016 3:02 AM
> > To: Steve Varnau
> > <st...@esgyn.com>>
> > Subject: about pull 503
> >
> > Hi Steve:
> >
> > I send this mail to discuss #503 (
> > https://github.com/apache/incubator-trafodion/pull/503)
> > As your comment is a little confused me, I’d like to make sure something.
> >
> > As u commented:
> >
> > In other cases, we support multiple versions simultaneously by
> > generating multiple TRX files. I'm not sure we want to add more TRX
> > files, but I'm also not sure we want to switch how the build works
> > based on content of install_features file.
> >
> > Previously it works the other way round -- the install_features file
> > indicates what TRX files are available, hence which distros are
> supported.
> >
> > This change will not add new trx file ,it just change
> > hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> > apache-trx.jar, because there are two reason, one is our install
> > script will add hbase-trx-apache*.jar to classpath, see
> > /incubator-trafodion/install/installer/traf_apache_mods
> > line51. Two is if 2 different version hbase-trx-apache*.jar added to
> > classpath, hbase will only use the first meet class in the jar
> > package, it may lead to conflict.
> >
> > About install_features file. I am not sure we want to switch how the
> > build works based on content of install_features file either. But the
> > only way I can distinguish which hbase version to use is this file. If
> > there is another place to distinguish this ,pls tell me ,thanks.
> >
> > About your last sentence , I am really confuse, is that means we
> > indeed use install_features to distinguish hbase version?
> >
> >
> > Thanks,
> > Aven
> >
> >
>
>
> --
> Thanks,
>
> Amanda Moran
>
--
Thanks,
Amanda Moran
RE: discuss on support hbase 1.1
Posted by Selva Govindarajan <se...@esgyn.com>.
My understanding is that install_features file lists the Trafodion required
products and its version. At the time of install, if the dependent product
listed in this file doesn't exist or the version below exists in the
cluster, install will fail.
With the change in sqenvcom.sh, you are determining which version of Trx-jar
to be built for apache hbase version. This would mean,
Change install_features to build Apache_1.0, build trx jar files correspond
to Apache hbase 1.0. And then change install_features to build Apache_1.1,
build trx jar files correspond to Apache hbase 1.1.
Trafodion build process doesn't support this concept.
So, it narrows down the question of should Trafodion support both Apache
Hbase 1.0 and Apache Hbase 1.1 from the same code base. Or is it ok to say
Trafodion 2.0 version needs Apache hbase 1.0 and Trafodion 2.1 version needs
Apache hbase 1.1.
My vote is for the latter option.
If so, basically you just need to remove the change in sqenvcom.sh and
rework it to build for Apache hbase 1.1.
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Tuesday, June 21, 2016 2:43 AM
To: dev@trafodion.incubator.apache.org
Subject: 答复: discuss on support hbase 1.1
Hi
Seemingly it's not a good solution to use install_features to determine
which trx file will use. I 'm try to add more trx files. So that
install_features file will just indicate what trx files are available
-----邮件原件-----
发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
发送时间: 2016年6月21日 13:15
收件人: dev@trafodion.incubator.apache.org
主题: 答复: discuss on support hbase 1.1
Quote Steve's view first
"In other cases, we support multiple versions simultaneously by generating
multiple TRX files. I'm not sure we want to add more TRX files, but I'm also
not sure we want to switch how the build works based on content of
install_features file.
Previously it works the other way round -- the install_features file
indicates what TRX files are available, hence which distros are supported."
No more trx files will be added.
Have to use install_features to decide whice hbase version will be
used.(discussed in the previous msg)
-----邮件原件-----
发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
发送时间: 2016年6月21日 12:51
收件人: dev@trafodion.incubator.apache.org
主题: RE: discuss on support hbase 1.1
I have the same observation as that of Steve for PR 503. Can you please
address them?
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Monday, June 20, 2016 9:00 PM
To: dev@trafodion.incubator.apache.org
Subject: 答复: discuss on support hbase 1.1
Hi All:
Can someone review #503
(https://github.com/apache/incubator-trafodion/pull/503 )?
Since there is a long time ,and several time discussion shows it's the only
solution.
Thanks
-----邮件原件-----
发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
发送时间: 2016年6月16日 10:51
收件人: dev@trafodion.incubator.apache.org
主题: 答复: discuss on support hbase 1.1
Yes reflection is a good option, but I have do some research , it's hard to
use reflection,
eg: TrxTransactionState.java need add @Override to next(..) for hbase 1.1,
and this I think can't use reflect #ifndef HDP2.3 HBASE1.1
@Override
#endif
public synchronized boolean next(final List<Cell> results) throws
IOException {
return next(results, -1);
}
Another is extends from new class, ScheduledChore is new class in hbase1.1
while Chore is disappear in hbase 1.0 #ifdef HDP2.3 HBASE1.1 public class
CleanOldTransactionsChore extends ScheduledChore { #else public class
CleanOldTransactionsChore extends Chore { #endif
And there still some kind of changes I can hardly use reflection...
-----邮件原件-----
发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
发送时间: 2016年6月15日 22:36
收件人: dev@trafodion.incubator.apache.org
主题: RE: discuss on support hbase 1.1
Another option is to check the code where Trafodion issues different APIs
and rework so that common set of API can be used. Reflection concept in java
might also work in some cases. If the rework is not possible then it might
be necessary to stick with higher version.
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Wednesday, June 15, 2016 2:14 AM
To: dev@trafodion.incubator.apache.org
Subject: discuss on support hbase 1.1
Hi all:
Apologize for this taking me so long... and thanks Amanda's reply (your
reply gives me confidence :)
Last time I had an inspiration to solve support hbase 1.1 without use
install_features, but the truth is it's a misconception...
I'd like to summary it first, according to the manual (see
https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall
)
:
1. download source code.
2. source sqenvr.sh & make package-all
3. install Hadoop & hbase
4. install trafodion
Base on the above steps, there is no idea knows which hbase version the user
will install. I can't mandatory provisions install Hadoop & hbase before
make trafodion package...
So the only way is to predefine hbase version in a config file, so there
appear the install_features.
Another question is why not support both hbase 1.0 & 1.1 at the same time?
The answer is hbase1.0 &1.1 have different api , so I can only support one
version at one package.
Further question is why not package two package eg: hbase-trx-apache1.0.jar
& hbase-trx-apache1.1.jar
It seems a solution, but I don't think it's a good way, reason is similar
why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
So I think this release the apache hbase ,we support 1.0...the next release
we support 1.1
====================================================================
All above is my view, maybe not correct... if some one good at this pls
correct.
Thanks.
-----邮件原件-----
发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
发送时间: 2016年6月4日 2:02
收件人: dev <de...@trafodion.incubator.apache.org>
抄送: Steve Varnau <st...@esgyn.com>
主题: Re: 答复: about pull 503
Hi there Aven-
You can you install_features, actually you must so that the installer knows
what version of HBase is supported in that particular package. The installer
can install all versions of trafodion. So there will need to be HBase 1.0
support in the installer and HBase 1.1 . install_features is used during
installation not during sqenvcom.sh. Take a look at traf_config_check to see
the details.
Thanks.
On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <shengchen.ma@esgyn.cn
> wrote:
> hi Steve:
> I think u are right . I will do some change which will not rely on
> install_feature. Then I will have this discussion on dev list.
>
> Thanks
> Aven
>
>
> 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> 发送时间: 2016年6月3日 6:40
> 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> 抄送: Amanda Moran <am...@esgyn.com>
> 主题: RE: about pull 503
>
> Hi Aven,
>
> I think it would be really good to have this discussion on the dev
> list or on github. Others may also be confused by my remarks or may
> have different opinions than I do. Maybe I’m wrong. Either way, it
> would be good record for next person trying to add support of another
> HBase version. But I did not want to post your message without
> permission, so I’ll reply directly.
>
> Your point #1 about installer script is a good one. But I‘d say that
> the installer script could be enhanced to recognize which version of
> HBase is there and then move only correct TRX file. Either as part of
> your change or separately.
>
> If you are changing build to support apache hbase1.1, then why are you
> trying to retain hbase1.0? If we want both supported, we should be
> able to choose at install time not at build time.
>
> It is hbase-trx makefile that determines which TRX files are built. If
> developr wants to switch hbase version, they should change makefile,
> and maybe the install_features file too. But the sqenvcom.sh file
> should not read the install_features file. Your changes work
> differently than all the other versions of TRX. I would prefer
> consistency.
>
> --Steve
>
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> shengchen.ma@esgyn.cn>]
> Sent: Thursday, June 2, 2016 3:02 AM
> To: Steve Varnau
> <st...@esgyn.com>>
> Subject: about pull 503
>
> Hi Steve:
>
> I send this mail to discuss #503 (
> https://github.com/apache/incubator-trafodion/pull/503)
> As your comment is a little confused me, I’d like to make sure something.
>
> As u commented:
>
> In other cases, we support multiple versions simultaneously by
> generating multiple TRX files. I'm not sure we want to add more TRX
> files, but I'm also not sure we want to switch how the build works
> based on content of install_features file.
>
> Previously it works the other way round -- the install_features file
> indicates what TRX files are available, hence which distros are supported.
>
> This change will not add new trx file ,it just change
> hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> apache-trx.jar, because there are two reason, one is our install
> script will add hbase-trx-apache*.jar to classpath, see
> /incubator-trafodion/install/installer/traf_apache_mods
> line51. Two is if 2 different version hbase-trx-apache*.jar added to
> classpath, hbase will only use the first meet class in the jar
> package, it may lead to conflict.
>
> About install_features file. I am not sure we want to switch how the
> build works based on content of install_features file either. But the
> only way I can distinguish which hbase version to use is this file. If
> there is another place to distinguish this ,pls tell me ,thanks.
>
> About your last sentence , I am really confuse, is that means we
> indeed use install_features to distinguish hbase version?
>
>
> Thanks,
> Aven
>
>
--
Thanks,
Amanda Moran
答复: discuss on support hbase 1.1
Posted by "Ma, Sheng-Chen (Aven)" <sh...@esgyn.cn>.
Hi
Seemingly it's not a good solution to use install_features to determine which trx file will use. I 'm try to add more trx files. So that install_features file will just indicate what trx files are available
-----邮件原件-----
发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
发送时间: 2016年6月21日 13:15
收件人: dev@trafodion.incubator.apache.org
主题: 答复: discuss on support hbase 1.1
Quote Steve's view first
"In other cases, we support multiple versions simultaneously by generating multiple TRX files. I'm not sure we want to add more TRX files, but I'm also not sure we want to switch how the build works based on content of install_features file.
Previously it works the other way round -- the install_features file indicates what TRX files are available, hence which distros are supported."
No more trx files will be added.
Have to use install_features to decide whice hbase version will be used.(discussed in the previous msg)
-----邮件原件-----
发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
发送时间: 2016年6月21日 12:51
收件人: dev@trafodion.incubator.apache.org
主题: RE: discuss on support hbase 1.1
I have the same observation as that of Steve for PR 503. Can you please address them?
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Monday, June 20, 2016 9:00 PM
To: dev@trafodion.incubator.apache.org
Subject: 答复: discuss on support hbase 1.1
Hi All:
Can someone review #503
(https://github.com/apache/incubator-trafodion/pull/503 )?
Since there is a long time ,and several time discussion shows it's the only solution.
Thanks
-----邮件原件-----
发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
发送时间: 2016年6月16日 10:51
收件人: dev@trafodion.incubator.apache.org
主题: 答复: discuss on support hbase 1.1
Yes reflection is a good option, but I have do some research , it's hard to use reflection,
eg: TrxTransactionState.java need add @Override to next(..) for hbase 1.1, and this I think can't use reflect #ifndef HDP2.3 HBASE1.1
@Override
#endif
public synchronized boolean next(final List<Cell> results) throws IOException {
return next(results, -1);
}
Another is extends from new class, ScheduledChore is new class in hbase1.1 while Chore is disappear in hbase 1.0 #ifdef HDP2.3 HBASE1.1 public class CleanOldTransactionsChore extends ScheduledChore { #else public class CleanOldTransactionsChore extends Chore { #endif
And there still some kind of changes I can hardly use reflection...
-----邮件原件-----
发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
发送时间: 2016年6月15日 22:36
收件人: dev@trafodion.incubator.apache.org
主题: RE: discuss on support hbase 1.1
Another option is to check the code where Trafodion issues different APIs and rework so that common set of API can be used. Reflection concept in java might also work in some cases. If the rework is not possible then it might be necessary to stick with higher version.
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Wednesday, June 15, 2016 2:14 AM
To: dev@trafodion.incubator.apache.org
Subject: discuss on support hbase 1.1
Hi all:
Apologize for this taking me so long... and thanks Amanda's reply (your reply gives me confidence :)
Last time I had an inspiration to solve support hbase 1.1 without use install_features, but the truth is it's a misconception...
I'd like to summary it first, according to the manual (see https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall
)
:
1. download source code.
2. source sqenvr.sh & make package-all
3. install Hadoop & hbase
4. install trafodion
Base on the above steps, there is no idea knows which hbase version the user will install. I can't mandatory provisions install Hadoop & hbase before make trafodion package...
So the only way is to predefine hbase version in a config file, so there appear the install_features.
Another question is why not support both hbase 1.0 & 1.1 at the same time?
The answer is hbase1.0 &1.1 have different api , so I can only support one version at one package.
Further question is why not package two package eg: hbase-trx-apache1.0.jar & hbase-trx-apache1.1.jar
It seems a solution, but I don't think it's a good way, reason is similar why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
So I think this release the apache hbase ,we support 1.0...the next release we support 1.1 ====================================================================
All above is my view, maybe not correct... if some one good at this pls correct.
Thanks.
-----邮件原件-----
发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
发送时间: 2016年6月4日 2:02
收件人: dev <de...@trafodion.incubator.apache.org>
抄送: Steve Varnau <st...@esgyn.com>
主题: Re: 答复: about pull 503
Hi there Aven-
You can you install_features, actually you must so that the installer knows what version of HBase is supported in that particular package. The installer can install all versions of trafodion. So there will need to be HBase 1.0 support in the installer and HBase 1.1 . install_features is used during installation not during sqenvcom.sh. Take a look at traf_config_check to see the details.
Thanks.
On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <shengchen.ma@esgyn.cn
> wrote:
> hi Steve:
> I think u are right . I will do some change which will not rely on
> install_feature. Then I will have this discussion on dev list.
>
> Thanks
> Aven
>
>
> 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> 发送时间: 2016年6月3日 6:40
> 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> 抄送: Amanda Moran <am...@esgyn.com>
> 主题: RE: about pull 503
>
> Hi Aven,
>
> I think it would be really good to have this discussion on the dev
> list or on github. Others may also be confused by my remarks or may
> have different opinions than I do. Maybe I’m wrong. Either way, it
> would be good record for next person trying to add support of another
> HBase version. But I did not want to post your message without
> permission, so I’ll reply directly.
>
> Your point #1 about installer script is a good one. But I‘d say that
> the installer script could be enhanced to recognize which version of
> HBase is there and then move only correct TRX file. Either as part of
> your change or separately.
>
> If you are changing build to support apache hbase1.1, then why are you
> trying to retain hbase1.0? If we want both supported, we should be
> able to choose at install time not at build time.
>
> It is hbase-trx makefile that determines which TRX files are built. If
> developr wants to switch hbase version, they should change makefile,
> and maybe the install_features file too. But the sqenvcom.sh file
> should not read the install_features file. Your changes work
> differently than all the other versions of TRX. I would prefer
> consistency.
>
> --Steve
>
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> shengchen.ma@esgyn.cn>]
> Sent: Thursday, June 2, 2016 3:02 AM
> To: Steve Varnau
> <st...@esgyn.com>>
> Subject: about pull 503
>
> Hi Steve:
>
> I send this mail to discuss #503 (
> https://github.com/apache/incubator-trafodion/pull/503)
> As your comment is a little confused me, I’d like to make sure something.
>
> As u commented:
>
> In other cases, we support multiple versions simultaneously by
> generating multiple TRX files. I'm not sure we want to add more TRX
> files, but I'm also not sure we want to switch how the build works
> based on content of install_features file.
>
> Previously it works the other way round -- the install_features file
> indicates what TRX files are available, hence which distros are supported.
>
> This change will not add new trx file ,it just change
> hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> apache-trx.jar, because there are two reason, one is our install
> script will add hbase-trx-apache*.jar to classpath, see
> /incubator-trafodion/install/installer/traf_apache_mods
> line51. Two is if 2 different version hbase-trx-apache*.jar added to
> classpath, hbase will only use the first meet class in the jar
> package, it may lead to conflict.
>
> About install_features file. I am not sure we want to switch how the
> build works based on content of install_features file either. But the
> only way I can distinguish which hbase version to use is this file. If
> there is another place to distinguish this ,pls tell me ,thanks.
>
> About your last sentence , I am really confuse, is that means we
> indeed use install_features to distinguish hbase version?
>
>
> Thanks,
> Aven
>
>
--
Thanks,
Amanda Moran
答复: discuss on support hbase 1.1
Posted by "Ma, Sheng-Chen (Aven)" <sh...@esgyn.cn>.
Quote Steve's view first
"In other cases, we support multiple versions simultaneously by generating multiple TRX files. I'm not sure we want to add more TRX files, but I'm also not sure we want to switch how the build works based on content of install_features file.
Previously it works the other way round -- the install_features file indicates what TRX files are available, hence which distros are supported."
No more trx files will be added.
Have to use install_features to decide whice hbase version will be used.(discussed in the previous msg)
-----邮件原件-----
发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
发送时间: 2016年6月21日 12:51
收件人: dev@trafodion.incubator.apache.org
主题: RE: discuss on support hbase 1.1
I have the same observation as that of Steve for PR 503. Can you please address them?
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Monday, June 20, 2016 9:00 PM
To: dev@trafodion.incubator.apache.org
Subject: 答复: discuss on support hbase 1.1
Hi All:
Can someone review #503
(https://github.com/apache/incubator-trafodion/pull/503 )?
Since there is a long time ,and several time discussion shows it's the only solution.
Thanks
-----邮件原件-----
发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
发送时间: 2016年6月16日 10:51
收件人: dev@trafodion.incubator.apache.org
主题: 答复: discuss on support hbase 1.1
Yes reflection is a good option, but I have do some research , it's hard to use reflection,
eg: TrxTransactionState.java need add @Override to next(..) for hbase 1.1, and this I think can't use reflect #ifndef HDP2.3 HBASE1.1
@Override
#endif
public synchronized boolean next(final List<Cell> results) throws IOException {
return next(results, -1);
}
Another is extends from new class, ScheduledChore is new class in hbase1.1 while Chore is disappear in hbase 1.0 #ifdef HDP2.3 HBASE1.1 public class CleanOldTransactionsChore extends ScheduledChore { #else public class CleanOldTransactionsChore extends Chore { #endif
And there still some kind of changes I can hardly use reflection...
-----邮件原件-----
发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
发送时间: 2016年6月15日 22:36
收件人: dev@trafodion.incubator.apache.org
主题: RE: discuss on support hbase 1.1
Another option is to check the code where Trafodion issues different APIs and rework so that common set of API can be used. Reflection concept in java might also work in some cases. If the rework is not possible then it might be necessary to stick with higher version.
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Wednesday, June 15, 2016 2:14 AM
To: dev@trafodion.incubator.apache.org
Subject: discuss on support hbase 1.1
Hi all:
Apologize for this taking me so long... and thanks Amanda's reply (your reply gives me confidence :)
Last time I had an inspiration to solve support hbase 1.1 without use install_features, but the truth is it's a misconception...
I'd like to summary it first, according to the manual (see https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall
)
:
1. download source code.
2. source sqenvr.sh & make package-all
3. install Hadoop & hbase
4. install trafodion
Base on the above steps, there is no idea knows which hbase version the user will install. I can't mandatory provisions install Hadoop & hbase before make trafodion package...
So the only way is to predefine hbase version in a config file, so there appear the install_features.
Another question is why not support both hbase 1.0 & 1.1 at the same time?
The answer is hbase1.0 &1.1 have different api , so I can only support one version at one package.
Further question is why not package two package eg: hbase-trx-apache1.0.jar & hbase-trx-apache1.1.jar
It seems a solution, but I don't think it's a good way, reason is similar why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
So I think this release the apache hbase ,we support 1.0...the next release we support 1.1 ====================================================================
All above is my view, maybe not correct... if some one good at this pls correct.
Thanks.
-----邮件原件-----
发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
发送时间: 2016年6月4日 2:02
收件人: dev <de...@trafodion.incubator.apache.org>
抄送: Steve Varnau <st...@esgyn.com>
主题: Re: 答复: about pull 503
Hi there Aven-
You can you install_features, actually you must so that the installer knows what version of HBase is supported in that particular package. The installer can install all versions of trafodion. So there will need to be HBase 1.0 support in the installer and HBase 1.1 . install_features is used during installation not during sqenvcom.sh. Take a look at traf_config_check to see the details.
Thanks.
On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <shengchen.ma@esgyn.cn
> wrote:
> hi Steve:
> I think u are right . I will do some change which will not rely on
> install_feature. Then I will have this discussion on dev list.
>
> Thanks
> Aven
>
>
> 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> 发送时间: 2016年6月3日 6:40
> 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> 抄送: Amanda Moran <am...@esgyn.com>
> 主题: RE: about pull 503
>
> Hi Aven,
>
> I think it would be really good to have this discussion on the dev
> list or on github. Others may also be confused by my remarks or may
> have different opinions than I do. Maybe I’m wrong. Either way, it
> would be good record for next person trying to add support of another
> HBase version. But I did not want to post your message without
> permission, so I’ll reply directly.
>
> Your point #1 about installer script is a good one. But I‘d say that
> the installer script could be enhanced to recognize which version of
> HBase is there and then move only correct TRX file. Either as part of
> your change or separately.
>
> If you are changing build to support apache hbase1.1, then why are you
> trying to retain hbase1.0? If we want both supported, we should be
> able to choose at install time not at build time.
>
> It is hbase-trx makefile that determines which TRX files are built. If
> developr wants to switch hbase version, they should change makefile,
> and maybe the install_features file too. But the sqenvcom.sh file
> should not read the install_features file. Your changes work
> differently than all the other versions of TRX. I would prefer
> consistency.
>
> --Steve
>
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> shengchen.ma@esgyn.cn>]
> Sent: Thursday, June 2, 2016 3:02 AM
> To: Steve Varnau
> <st...@esgyn.com>>
> Subject: about pull 503
>
> Hi Steve:
>
> I send this mail to discuss #503 (
> https://github.com/apache/incubator-trafodion/pull/503)
> As your comment is a little confused me, I’d like to make sure something.
>
> As u commented:
>
> In other cases, we support multiple versions simultaneously by
> generating multiple TRX files. I'm not sure we want to add more TRX
> files, but I'm also not sure we want to switch how the build works
> based on content of install_features file.
>
> Previously it works the other way round -- the install_features file
> indicates what TRX files are available, hence which distros are supported.
>
> This change will not add new trx file ,it just change
> hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> apache-trx.jar, because there are two reason, one is our install
> script will add hbase-trx-apache*.jar to classpath, see
> /incubator-trafodion/install/installer/traf_apache_mods
> line51. Two is if 2 different version hbase-trx-apache*.jar added to
> classpath, hbase will only use the first meet class in the jar
> package, it may lead to conflict.
>
> About install_features file. I am not sure we want to switch how the
> build works based on content of install_features file either. But the
> only way I can distinguish which hbase version to use is this file. If
> there is another place to distinguish this ,pls tell me ,thanks.
>
> About your last sentence , I am really confuse, is that means we
> indeed use install_features to distinguish hbase version?
>
>
> Thanks,
> Aven
>
>
--
Thanks,
Amanda Moran
RE: discuss on support hbase 1.1
Posted by Selva Govindarajan <se...@esgyn.com>.
I have the same observation as that of Steve for PR 503. Can you please
address them?
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Monday, June 20, 2016 9:00 PM
To: dev@trafodion.incubator.apache.org
Subject: 答复: discuss on support hbase 1.1
Hi All:
Can someone review #503
(https://github.com/apache/incubator-trafodion/pull/503 )?
Since there is a long time ,and several time discussion shows it's the only
solution.
Thanks
-----邮件原件-----
发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
发送时间: 2016年6月16日 10:51
收件人: dev@trafodion.incubator.apache.org
主题: 答复: discuss on support hbase 1.1
Yes reflection is a good option, but I have do some research , it's hard to
use reflection,
eg: TrxTransactionState.java need add @Override to next(..) for hbase 1.1,
and this I think can't use reflect #ifndef HDP2.3 HBASE1.1
@Override
#endif
public synchronized boolean next(final List<Cell> results) throws
IOException {
return next(results, -1);
}
Another is extends from new class, ScheduledChore is new class in hbase1.1
while Chore is disappear in hbase 1.0 #ifdef HDP2.3 HBASE1.1 public class
CleanOldTransactionsChore extends ScheduledChore { #else public class
CleanOldTransactionsChore extends Chore { #endif
And there still some kind of changes I can hardly use reflection...
-----邮件原件-----
发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
发送时间: 2016年6月15日 22:36
收件人: dev@trafodion.incubator.apache.org
主题: RE: discuss on support hbase 1.1
Another option is to check the code where Trafodion issues different APIs
and rework so that common set of API can be used. Reflection concept in java
might also work in some cases. If the rework is not possible then it might
be necessary to stick with higher version.
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Wednesday, June 15, 2016 2:14 AM
To: dev@trafodion.incubator.apache.org
Subject: discuss on support hbase 1.1
Hi all:
Apologize for this taking me so long... and thanks Amanda's reply (your
reply gives me confidence :)
Last time I had an inspiration to solve support hbase 1.1 without use
install_features, but the truth is it's a misconception...
I'd like to summary it first, according to the manual (see
https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall
)
:
1. download source code.
2. source sqenvr.sh & make package-all
3. install Hadoop & hbase
4. install trafodion
Base on the above steps, there is no idea knows which hbase version the user
will install. I can't mandatory provisions install Hadoop & hbase before
make trafodion package...
So the only way is to predefine hbase version in a config file, so there
appear the install_features.
Another question is why not support both hbase 1.0 & 1.1 at the same time?
The answer is hbase1.0 &1.1 have different api , so I can only support one
version at one package.
Further question is why not package two package eg: hbase-trx-apache1.0.jar
& hbase-trx-apache1.1.jar
It seems a solution, but I don't think it's a good way, reason is similar
why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
So I think this release the apache hbase ,we support 1.0...the next release
we support 1.1
====================================================================
All above is my view, maybe not correct... if some one good at this pls
correct.
Thanks.
-----邮件原件-----
发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
发送时间: 2016年6月4日 2:02
收件人: dev <de...@trafodion.incubator.apache.org>
抄送: Steve Varnau <st...@esgyn.com>
主题: Re: 答复: about pull 503
Hi there Aven-
You can you install_features, actually you must so that the installer knows
what version of HBase is supported in that particular package. The installer
can install all versions of trafodion. So there will need to be HBase 1.0
support in the installer and HBase 1.1 . install_features is used during
installation not during sqenvcom.sh. Take a look at traf_config_check to see
the details.
Thanks.
On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <shengchen.ma@esgyn.cn
> wrote:
> hi Steve:
> I think u are right . I will do some change which will not rely on
> install_feature. Then I will have this discussion on dev list.
>
> Thanks
> Aven
>
>
> 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> 发送时间: 2016年6月3日 6:40
> 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> 抄送: Amanda Moran <am...@esgyn.com>
> 主题: RE: about pull 503
>
> Hi Aven,
>
> I think it would be really good to have this discussion on the dev
> list or on github. Others may also be confused by my remarks or may
> have different opinions than I do. Maybe I’m wrong. Either way, it
> would be good record for next person trying to add support of another
> HBase version. But I did not want to post your message without
> permission, so I’ll reply directly.
>
> Your point #1 about installer script is a good one. But I‘d say that
> the installer script could be enhanced to recognize which version of
> HBase is there and then move only correct TRX file. Either as part of
> your change or separately.
>
> If you are changing build to support apache hbase1.1, then why are you
> trying to retain hbase1.0? If we want both supported, we should be
> able to choose at install time not at build time.
>
> It is hbase-trx makefile that determines which TRX files are built. If
> developr wants to switch hbase version, they should change makefile,
> and maybe the install_features file too. But the sqenvcom.sh file
> should not read the install_features file. Your changes work
> differently than all the other versions of TRX. I would prefer
> consistency.
>
> --Steve
>
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> shengchen.ma@esgyn.cn>]
> Sent: Thursday, June 2, 2016 3:02 AM
> To: Steve Varnau
> <st...@esgyn.com>>
> Subject: about pull 503
>
> Hi Steve:
>
> I send this mail to discuss #503 (
> https://github.com/apache/incubator-trafodion/pull/503)
> As your comment is a little confused me, I’d like to make sure something.
>
> As u commented:
>
> In other cases, we support multiple versions simultaneously by
> generating multiple TRX files. I'm not sure we want to add more TRX
> files, but I'm also not sure we want to switch how the build works
> based on content of install_features file.
>
> Previously it works the other way round -- the install_features file
> indicates what TRX files are available, hence which distros are supported.
>
> This change will not add new trx file ,it just change
> hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> apache-trx.jar, because there are two reason, one is our install
> script will add hbase-trx-apache*.jar to classpath, see
> /incubator-trafodion/install/installer/traf_apache_mods
> line51. Two is if 2 different version hbase-trx-apache*.jar added to
> classpath, hbase will only use the first meet class in the jar
> package, it may lead to conflict.
>
> About install_features file. I am not sure we want to switch how the
> build works based on content of install_features file either. But the
> only way I can distinguish which hbase version to use is this file. If
> there is another place to distinguish this ,pls tell me ,thanks.
>
> About your last sentence , I am really confuse, is that means we
> indeed use install_features to distinguish hbase version?
>
>
> Thanks,
> Aven
>
>
--
Thanks,
Amanda Moran
答复: discuss on support hbase 1.1
Posted by "Ma, Sheng-Chen (Aven)" <sh...@esgyn.cn>.
Hi All:
Can someone review #503 (https://github.com/apache/incubator-trafodion/pull/503 )?
Since there is a long time ,and several time discussion shows it's the only solution.
Thanks
-----邮件原件-----
发件人: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
发送时间: 2016年6月16日 10:51
收件人: dev@trafodion.incubator.apache.org
主题: 答复: discuss on support hbase 1.1
Yes reflection is a good option, but I have do some research , it's hard to use reflection,
eg: TrxTransactionState.java need add @Override to next(..) for hbase 1.1, and this I think can't use reflect #ifndef HDP2.3 HBASE1.1
@Override
#endif
public synchronized boolean next(final List<Cell> results) throws IOException {
return next(results, -1);
}
Another is extends from new class, ScheduledChore is new class in hbase1.1 while Chore is disappear in hbase 1.0 #ifdef HDP2.3 HBASE1.1 public class CleanOldTransactionsChore extends ScheduledChore { #else public class CleanOldTransactionsChore extends Chore { #endif
And there still some kind of changes I can hardly use reflection...
-----邮件原件-----
发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
发送时间: 2016年6月15日 22:36
收件人: dev@trafodion.incubator.apache.org
主题: RE: discuss on support hbase 1.1
Another option is to check the code where Trafodion issues different APIs and rework so that common set of API can be used. Reflection concept in java might also work in some cases. If the rework is not possible then it might be necessary to stick with higher version.
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Wednesday, June 15, 2016 2:14 AM
To: dev@trafodion.incubator.apache.org
Subject: discuss on support hbase 1.1
Hi all:
Apologize for this taking me so long... and thanks Amanda's reply (your reply gives me confidence :)
Last time I had an inspiration to solve support hbase 1.1 without use install_features, but the truth is it's a misconception...
I'd like to summary it first, according to the manual (see https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall
)
:
1. download source code.
2. source sqenvr.sh & make package-all
3. install Hadoop & hbase
4. install trafodion
Base on the above steps, there is no idea knows which hbase version the user will install. I can't mandatory provisions install Hadoop & hbase before make trafodion package...
So the only way is to predefine hbase version in a config file, so there appear the install_features.
Another question is why not support both hbase 1.0 & 1.1 at the same time?
The answer is hbase1.0 &1.1 have different api , so I can only support one version at one package.
Further question is why not package two package eg: hbase-trx-apache1.0.jar & hbase-trx-apache1.1.jar
It seems a solution, but I don't think it's a good way, reason is similar why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
So I think this release the apache hbase ,we support 1.0...the next release we support 1.1 ====================================================================
All above is my view, maybe not correct... if some one good at this pls correct.
Thanks.
-----邮件原件-----
发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
发送时间: 2016年6月4日 2:02
收件人: dev <de...@trafodion.incubator.apache.org>
抄送: Steve Varnau <st...@esgyn.com>
主题: Re: 答复: about pull 503
Hi there Aven-
You can you install_features, actually you must so that the installer knows what version of HBase is supported in that particular package. The installer can install all versions of trafodion. So there will need to be HBase 1.0 support in the installer and HBase 1.1 . install_features is used during installation not during sqenvcom.sh. Take a look at traf_config_check to see the details.
Thanks.
On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <shengchen.ma@esgyn.cn
> wrote:
> hi Steve:
> I think u are right . I will do some change which will not rely on
> install_feature. Then I will have this discussion on dev list.
>
> Thanks
> Aven
>
>
> 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> 发送时间: 2016年6月3日 6:40
> 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> 抄送: Amanda Moran <am...@esgyn.com>
> 主题: RE: about pull 503
>
> Hi Aven,
>
> I think it would be really good to have this discussion on the dev
> list or on github. Others may also be confused by my remarks or may
> have different opinions than I do. Maybe I’m wrong. Either way, it
> would be good record for next person trying to add support of another
> HBase version. But I did not want to post your message without
> permission, so I’ll reply directly.
>
> Your point #1 about installer script is a good one. But I‘d say that
> the installer script could be enhanced to recognize which version of
> HBase is there and then move only correct TRX file. Either as part of
> your change or separately.
>
> If you are changing build to support apache hbase1.1, then why are you
> trying to retain hbase1.0? If we want both supported, we should be
> able to choose at install time not at build time.
>
> It is hbase-trx makefile that determines which TRX files are built. If
> developr wants to switch hbase version, they should change makefile,
> and maybe the install_features file too. But the sqenvcom.sh file
> should not read the install_features file. Your changes work
> differently than all the other versions of TRX. I would prefer
> consistency.
>
> --Steve
>
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> shengchen.ma@esgyn.cn>]
> Sent: Thursday, June 2, 2016 3:02 AM
> To: Steve Varnau
> <st...@esgyn.com>>
> Subject: about pull 503
>
> Hi Steve:
>
> I send this mail to discuss #503 (
> https://github.com/apache/incubator-trafodion/pull/503)
> As your comment is a little confused me, I’d like to make sure something.
>
> As u commented:
>
> In other cases, we support multiple versions simultaneously by
> generating multiple TRX files. I'm not sure we want to add more TRX
> files, but I'm also not sure we want to switch how the build works
> based on content of install_features file.
>
> Previously it works the other way round -- the install_features file
> indicates what TRX files are available, hence which distros are supported.
>
> This change will not add new trx file ,it just change
> hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> apache-trx.jar, because there are two reason, one is our install
> script will add hbase-trx-apache*.jar to classpath, see
> /incubator-trafodion/install/installer/traf_apache_mods
> line51. Two is if 2 different version hbase-trx-apache*.jar added to
> classpath, hbase will only use the first meet class in the jar
> package, it may lead to conflict.
>
> About install_features file. I am not sure we want to switch how the
> build works based on content of install_features file either. But the
> only way I can distinguish which hbase version to use is this file. If
> there is another place to distinguish this ,pls tell me ,thanks.
>
> About your last sentence , I am really confuse, is that means we
> indeed use install_features to distinguish hbase version?
>
>
> Thanks,
> Aven
>
>
--
Thanks,
Amanda Moran
答复: discuss on support hbase 1.1
Posted by "Ma, Sheng-Chen (Aven)" <sh...@esgyn.cn>.
Yes reflection is a good option, but I have do some research , it's hard to use reflection,
eg: TrxTransactionState.java need add @Override to next(..) for hbase 1.1, and this I think can't use reflect
#ifndef HDP2.3 HBASE1.1
@Override
#endif
public synchronized boolean next(final List<Cell> results) throws IOException {
return next(results, -1);
}
Another is extends from new class, ScheduledChore is new class in hbase1.1 while Chore is disappear in hbase 1.0
#ifdef HDP2.3 HBASE1.1
public class CleanOldTransactionsChore extends ScheduledChore {
#else
public class CleanOldTransactionsChore extends Chore {
#endif
And there still some kind of changes I can hardly use reflection...
-----邮件原件-----
发件人: Selva Govindarajan [mailto:selva.govindarajan@esgyn.com]
发送时间: 2016年6月15日 22:36
收件人: dev@trafodion.incubator.apache.org
主题: RE: discuss on support hbase 1.1
Another option is to check the code where Trafodion issues different APIs and rework so that common set of API can be used. Reflection concept in java might also work in some cases. If the rework is not possible then it might be necessary to stick with higher version.
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Wednesday, June 15, 2016 2:14 AM
To: dev@trafodion.incubator.apache.org
Subject: discuss on support hbase 1.1
Hi all:
Apologize for this taking me so long... and thanks Amanda's reply (your reply gives me confidence :)
Last time I had an inspiration to solve support hbase 1.1 without use install_features, but the truth is it's a misconception...
I'd like to summary it first, according to the manual (see https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall
)
:
1. download source code.
2. source sqenvr.sh & make package-all
3. install Hadoop & hbase
4. install trafodion
Base on the above steps, there is no idea knows which hbase version the user will install. I can't mandatory provisions install Hadoop & hbase before make trafodion package...
So the only way is to predefine hbase version in a config file, so there appear the install_features.
Another question is why not support both hbase 1.0 & 1.1 at the same time?
The answer is hbase1.0 &1.1 have different api , so I can only support one version at one package.
Further question is why not package two package eg: hbase-trx-apache1.0.jar & hbase-trx-apache1.1.jar
It seems a solution, but I don't think it's a good way, reason is similar why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
So I think this release the apache hbase ,we support 1.0...the next release we support 1.1 ====================================================================
All above is my view, maybe not correct... if some one good at this pls correct.
Thanks.
-----邮件原件-----
发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
发送时间: 2016年6月4日 2:02
收件人: dev <de...@trafodion.incubator.apache.org>
抄送: Steve Varnau <st...@esgyn.com>
主题: Re: 答复: about pull 503
Hi there Aven-
You can you install_features, actually you must so that the installer knows what version of HBase is supported in that particular package. The installer can install all versions of trafodion. So there will need to be HBase 1.0 support in the installer and HBase 1.1 . install_features is used during installation not during sqenvcom.sh. Take a look at traf_config_check to see the details.
Thanks.
On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <shengchen.ma@esgyn.cn
> wrote:
> hi Steve:
> I think u are right . I will do some change which will not rely on
> install_feature. Then I will have this discussion on dev list.
>
> Thanks
> Aven
>
>
> 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> 发送时间: 2016年6月3日 6:40
> 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> 抄送: Amanda Moran <am...@esgyn.com>
> 主题: RE: about pull 503
>
> Hi Aven,
>
> I think it would be really good to have this discussion on the dev
> list or on github. Others may also be confused by my remarks or may
> have different opinions than I do. Maybe I’m wrong. Either way, it
> would be good record for next person trying to add support of another
> HBase version. But I did not want to post your message without
> permission, so I’ll reply directly.
>
> Your point #1 about installer script is a good one. But I‘d say that
> the installer script could be enhanced to recognize which version of
> HBase is there and then move only correct TRX file. Either as part of
> your change or separately.
>
> If you are changing build to support apache hbase1.1, then why are you
> trying to retain hbase1.0? If we want both supported, we should be
> able to choose at install time not at build time.
>
> It is hbase-trx makefile that determines which TRX files are built. If
> developr wants to switch hbase version, they should change makefile,
> and maybe the install_features file too. But the sqenvcom.sh file
> should not read the install_features file. Your changes work
> differently than all the other versions of TRX. I would prefer
> consistency.
>
> --Steve
>
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> shengchen.ma@esgyn.cn>]
> Sent: Thursday, June 2, 2016 3:02 AM
> To: Steve Varnau
> <st...@esgyn.com>>
> Subject: about pull 503
>
> Hi Steve:
>
> I send this mail to discuss #503 (
> https://github.com/apache/incubator-trafodion/pull/503)
> As your comment is a little confused me, I’d like to make sure something.
>
> As u commented:
>
> In other cases, we support multiple versions simultaneously by
> generating multiple TRX files. I'm not sure we want to add more TRX
> files, but I'm also not sure we want to switch how the build works
> based on content of install_features file.
>
> Previously it works the other way round -- the install_features file
> indicates what TRX files are available, hence which distros are supported.
>
> This change will not add new trx file ,it just change
> hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> apache-trx.jar, because there are two reason, one is our install
> script will add hbase-trx-apache*.jar to classpath, see
> /incubator-trafodion/install/installer/traf_apache_mods
> line51. Two is if 2 different version hbase-trx-apache*.jar added to
> classpath, hbase will only use the first meet class in the jar
> package, it may lead to conflict.
>
> About install_features file. I am not sure we want to switch how the
> build works based on content of install_features file either. But the
> only way I can distinguish which hbase version to use is this file. If
> there is another place to distinguish this ,pls tell me ,thanks.
>
> About your last sentence , I am really confuse, is that means we
> indeed use install_features to distinguish hbase version?
>
>
> Thanks,
> Aven
>
>
--
Thanks,
Amanda Moran
RE: discuss on support hbase 1.1
Posted by Selva Govindarajan <se...@esgyn.com>.
Another option is to check the code where Trafodion issues different APIs
and rework so that common set of API can be used. Reflection concept in java
might also work in some cases. If the rework is not possible then it might
be necessary to stick with higher version.
Selva
-----Original Message-----
From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn]
Sent: Wednesday, June 15, 2016 2:14 AM
To: dev@trafodion.incubator.apache.org
Subject: discuss on support hbase 1.1
Hi all:
Apologize for this taking me so long... and thanks Amanda's reply (your
reply gives me confidence :)
Last time I had an inspiration to solve support hbase 1.1 without use
install_features, but the truth is it's a misconception...
I'd like to summary it first, according to the manual (see
https://cwiki.apache.org/confluence/display/TRAFODION/Build+Source#BuildSource-makeall
)
:
1. download source code.
2. source sqenvr.sh & make package-all
3. install Hadoop & hbase
4. install trafodion
Base on the above steps, there is no idea knows which hbase version the user
will install. I can't mandatory provisions install Hadoop & hbase before
make trafodion package...
So the only way is to predefine hbase version in a config file, so there
appear the install_features.
Another question is why not support both hbase 1.0 & 1.1 at the same time?
The answer is hbase1.0 &1.1 have different api , so I can only support one
version at one package.
Further question is why not package two package eg: hbase-trx-apache1.0.jar
& hbase-trx-apache1.1.jar
It seems a solution, but I don't think it's a good way, reason is similar
why there is only hbase-trx-cdh5_4.jar and hbase-trx-hdp2_3.jar.
So I think this release the apache hbase ,we support 1.0...the next release
we support 1.1
====================================================================
All above is my view, maybe not correct... if some one good at this pls
correct.
Thanks.
-----邮件原件-----
发件人: Amanda Moran [mailto:amanda.moran@esgyn.com]
发送时间: 2016年6月4日 2:02
收件人: dev <de...@trafodion.incubator.apache.org>
抄送: Steve Varnau <st...@esgyn.com>
主题: Re: 答复: about pull 503
Hi there Aven-
You can you install_features, actually you must so that the installer knows
what version of HBase is supported in that particular package. The installer
can install all versions of trafodion. So there will need to be HBase 1.0
support in the installer and HBase 1.1 . install_features is used during
installation not during sqenvcom.sh. Take a look at traf_config_check to see
the details.
Thanks.
On Fri, Jun 3, 2016 at 1:01 AM, Ma, Sheng-Chen (Aven) <shengchen.ma@esgyn.cn
> wrote:
> hi Steve:
> I think u are right . I will do some change which will not rely on
> install_feature. Then I will have this discussion on dev list.
>
> Thanks
> Aven
>
>
> 发件人: Steve Varnau [mailto:steve.varnau@esgyn.com]
> 发送时间: 2016年6月3日 6:40
> 收件人: Ma, Sheng-Chen (Aven) <sh...@esgyn.cn>
> 抄送: Amanda Moran <am...@esgyn.com>
> 主题: RE: about pull 503
>
> Hi Aven,
>
> I think it would be really good to have this discussion on the dev
> list or on github. Others may also be confused by my remarks or may
> have different opinions than I do. Maybe I’m wrong. Either way, it
> would be good record for next person trying to add support of another
> HBase version. But I did not want to post your message without permission,
> so I’ll reply directly.
>
> Your point #1 about installer script is a good one. But I‘d say that
> the installer script could be enhanced to recognize which version of
> HBase is there and then move only correct TRX file. Either as part of
> your change or separately.
>
> If you are changing build to support apache hbase1.1, then why are you
> trying to retain hbase1.0? If we want both supported, we should be
> able to choose at install time not at build time.
>
> It is hbase-trx makefile that determines which TRX files are built. If
> developr wants to switch hbase version, they should change makefile,
> and maybe the install_features file too. But the sqenvcom.sh file
> should not read the install_features file. Your changes work
> differently than all the other versions of TRX. I would prefer
> consistency.
>
> --Steve
>
> From: Ma, Sheng-Chen (Aven) [mailto:shengchen.ma@esgyn.cn<mailto:
> shengchen.ma@esgyn.cn>]
> Sent: Thursday, June 2, 2016 3:02 AM
> To: Steve Varnau
> <st...@esgyn.com>>
> Subject: about pull 503
>
> Hi Steve:
>
> I send this mail to discuss #503 (
> https://github.com/apache/incubator-trafodion/pull/503)
> As your comment is a little confused me, I’d like to make sure something.
>
> As u commented:
>
> In other cases, we support multiple versions simultaneously by
> generating multiple TRX files. I'm not sure we want to add more TRX
> files, but I'm also not sure we want to switch how the build works
> based on content of install_features file.
>
> Previously it works the other way round -- the install_features file
> indicates what TRX files are available, hence which distros are supported.
>
> This change will not add new trx file ,it just change
> hbase-trx-apache.jar from 1.0 to 1.1, It’s not good idea gen two
> apache-trx.jar, because there are two reason, one is our install
> script will add hbase-trx-apache*.jar to classpath, see
> /incubator-trafodion/install/installer/traf_apache_mods
> line51. Two is if 2 different version hbase-trx-apache*.jar added to
> classpath, hbase will only use the first meet class in the jar
> package, it may lead to conflict.
>
> About install_features file. I am not sure we want to switch how the
> build works based on content of install_features file either. But the
> only way I can distinguish which hbase version to use is this file. If
> there is another place to distinguish this ,pls tell me ,thanks.
>
> About your last sentence , I am really confuse, is that means we
> indeed use install_features to distinguish hbase version?
>
>
> Thanks,
> Aven
>
>
--
Thanks,
Amanda Moran