You are viewing a plain text version of this content. The canonical link for it is here.
Posted to codereview@trafodion.apache.org by sandhyasun <gi...@git.apache.org> on 2016/12/22 19:04:30 UTC
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
GitHub user sandhyasun opened a pull request:
https://github.com/apache/incubator-trafodion/pull/889
[TRAFODION-2408] Changes to support "empty lob" insert/update and update directly using lobhandle
Empty_blob()/ Emptyclob() functions will insert a dummy row into the LOB descriptor tables as a place holder. The lobhandle will be created but will contain no data. This lob handle can later be used to update data directly to it without scanning the SQL table.
JIRA contains more details on syntax/examples.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sandhyasun/incubator-trafodion empty_lob_work
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/889.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #889
----
commit 7baf325f3fd612125f49b6f6763e6e895ba52d58
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2016-12-21T19:04:25Z
Changes to support new syntax and implementation empty_blob/empty_clob for insert/updates
Changes to support new syntax for updating a lob directly through a lobhandle without scanning the table.This is done via new ExeUtil operator.
Changes to add columnname info to LOBMD_ table. This helps in mapping any given lobhandle back to the table and the column.
Added new sections to the regression test executor/TEST130 to test these changes
commit f364d03a5b73373421b885f73302545ad5208bc7
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2016-12-21T19:11:02Z
Merge remote branch 'origin/master' into empty_lob_work
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94463262
--- Diff: core/sql/optimizer/SynthType.cpp ---
@@ -6763,19 +6772,28 @@ const NAType *LOBupdate::synthesizeType()
{
// Return blob type
- ValueId vid1 = child(0)->getValueId();
- const NAType &typ1 = (NAType&)vid1.getType();
+ ValueId vid1,vid2 ;
+ const NAType *typ1,*typ2 = NULL;
- ValueId vid2 = child(1)->getValueId();
- const NAType &typ2 = (NAType&)vid2.getType();
+ if(child(0))
+ {
+ vid1= child(0)->getValueId();
+ typ1 = &vid1.getType();
+ }
+
+ if(child(1))
+ {
+ vid2 = child(1)->getValueId();
+ typ2 = &vid2.getType();
+ }
if ((obj_ == STRING_) ||
(obj_ == FILE_) ||
(obj_ == EXTERNAL_))
{
- if (typ1.getTypeQualifier() != NA_CHARACTER_TYPE)
+ if (typ1->getTypeQualifier() != NA_CHARACTER_TYPE)
--- End diff --
Added a check for that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94458775
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -267,3 +267,107 @@ Int32 updateAppendBufferToLob(CliGlobals *cliglob, char *tableName, char *column
return retcode;
}
+
+
+Int32 updateBufferToLobHandle(CliGlobals *cliglob,char *handle)
--- End diff --
#define LOB_HANDLE_LEN 1024
This is defined in ExpLOB.h
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94454952
--- Diff: core/sql/optimizer/SynthType.cpp ---
@@ -6699,16 +6699,20 @@ const NAType *LOBoper::synthesizeType()
const NAType *LOBinsert::synthesizeType()
{
// Return blob type
-
- ValueId vid1 = child(0)->getValueId();
- const NAType &typ1 = (NAType&)vid1.getType();
+ ValueId vid1;
+ const NAType *typ1 = NULL;
+ if (child(0))
+ {
+ vid1 = child(0)->getValueId();
+ typ1 = &vid1.getType();
+ }
if ((obj_ == STRING_) ||
(obj_ == FILE_) ||
(obj_ == EXTERNAL_) ||
(obj_ == LOAD_))
{
- if (typ1.getTypeQualifier() != NA_CHARACTER_TYPE)
+ if (typ1->getTypeQualifier() != NA_CHARACTER_TYPE)
--- End diff --
Could typ1 be null here? (Looks like it will be if child(0) is null.) Should we check for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94451027
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -267,3 +267,107 @@ Int32 updateAppendBufferToLob(CliGlobals *cliglob, char *tableName, char *column
return retcode;
}
+
+
+Int32 updateBufferToLobHandle(CliGlobals *cliglob,char *handle)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // update lob data into via a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
+ str_sprintf(query,"update lob (LOB '%s' , LOCATION %Ld, SIZE %Ld)", handle, (Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ {
+ cliInterface.executeImmediate("rollback work");
+ delete query;
+ delete lobDataBuf;
+ return retcode;
+ }
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+Int32 updateAppendBufferToLobHandle(CliGlobals *cliglob,char *handle)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // update lob data into via a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
--- End diff --
Buffer overrun here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94454033
--- Diff: core/sql/exp/ExpLOBaccess.cpp ---
@@ -1502,7 +1507,50 @@ Ex_Lob_Error ExLob::allocateDesc(ULng32 size, Int64 &descNum, Int64 &dataOffset,
char logBuf[4096];
lobDebugInfo("In ExLob::allocateDesc",0,__LINE__,lobTrace_);
Int32 openFlags = O_RDONLY ;
-
+ if (size == 0) //we are trying to empty this lob.
+ {
+ //rename lob datafile
+ char * saveLobDataFile = new(getLobGlobalHeap()) char[MAX_LOB_FILE_NAME_LEN+6];
+ str_sprintf(saveLobDataFile, "%s_save",lobDataFile_);
+ Int32 rc2 = hdfsRename(fs_,lobDataFile_,saveLobDataFile);
+ if (rc2 == -1)
+ {
+ lobDebugInfo("Problem renaming datafile to save data file",0,__LINE__,lobTrace_);
+ NADELETEBASIC(saveLobDataFile,getLobGlobalHeap());
+ return LOB_DATA_FILE_WRITE_ERROR;
+ }
+ //create a new file of the same name.
+ hdfsFile fdNew = hdfsOpenFile(fs_, lobDataFile_,O_WRONLY|O_CREAT,0,0,0);
+ if (!fdNew)
+ {
+ str_sprintf(logBuf,"Could not create/open file:%s",lobDataFile_);
+ lobDebugInfo(logBuf,0,__LINE__,lobTrace_);
+
+ //restore previous version
+ Int32 rc2 = hdfsRename(fs_,saveLobDataFile,lobDataFile_);
+ if (rc2 == -1)
+ {
+ lobDebugInfo("Problem restoring datafile . Will need to retry the update",0,__LINE__,lobTrace_);
+ NADELETEBASIC(saveLobDataFile,getLobGlobalHeap());
+ return LOB_DATA_FILE_WRITE_ERROR;
+ }
+ return LOB_DATA_FILE_OPEN_ERROR;
--- End diff --
How does saveLobDataFile get deleted if rc2 is not -1? Is there a memory leak here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94459025
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -267,3 +267,107 @@ Int32 updateAppendBufferToLob(CliGlobals *cliglob, char *tableName, char *column
return retcode;
}
+
+
+Int32 updateBufferToLobHandle(CliGlobals *cliglob,char *handle)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // update lob data into via a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
+ str_sprintf(query,"update lob (LOB '%s' , LOCATION %Ld, SIZE %Ld)", handle, (Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ {
+ cliInterface.executeImmediate("rollback work");
+ delete query;
+ delete lobDataBuf;
+ return retcode;
+ }
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+Int32 updateAppendBufferToLobHandle(CliGlobals *cliglob,char *handle)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // update lob data into via a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
--- End diff --
Here too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94461094
--- Diff: core/sql/exp/ExpLOBaccess.cpp ---
@@ -1502,7 +1507,50 @@ Ex_Lob_Error ExLob::allocateDesc(ULng32 size, Int64 &descNum, Int64 &dataOffset,
char logBuf[4096];
lobDebugInfo("In ExLob::allocateDesc",0,__LINE__,lobTrace_);
Int32 openFlags = O_RDONLY ;
-
+ if (size == 0) //we are trying to empty this lob.
+ {
+ //rename lob datafile
+ char * saveLobDataFile = new(getLobGlobalHeap()) char[MAX_LOB_FILE_NAME_LEN+6];
--- End diff --
This is fine. We only need to allocate for "_save" suffix
str_sprintf(saveLobDataFile, "%s_save",lobDataFile_);
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94460117
--- Diff: core/sql/exp/ExpLOB.cpp ---
@@ -1329,7 +1350,60 @@ ex_expr::exp_return_type ExpLOBupdate::eval(char *op_data[],
memcpy(&lobLen, op_data[3],sizeof(Int64)); // user specified buffer length
memcpy(data,op_data[1],sizeof(Int64)); // user buffer address
}
- if (isAppend())
+
+ if(fromEmpty())
+ {
+ lobLen = 0;
+ so = Lob_Memory;
+ }
+ /* if (fromEmpty())
--- End diff --
prototype code - removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94467477
--- Diff: core/sql/exp/ExpLOBaccess.cpp ---
@@ -1502,7 +1507,50 @@ Ex_Lob_Error ExLob::allocateDesc(ULng32 size, Int64 &descNum, Int64 &dataOffset,
char logBuf[4096];
lobDebugInfo("In ExLob::allocateDesc",0,__LINE__,lobTrace_);
Int32 openFlags = O_RDONLY ;
-
+ if (size == 0) //we are trying to empty this lob.
+ {
+ //rename lob datafile
+ char * saveLobDataFile = new(getLobGlobalHeap()) char[MAX_LOB_FILE_NAME_LEN+6];
+ str_sprintf(saveLobDataFile, "%s_save",lobDataFile_);
+ Int32 rc2 = hdfsRename(fs_,lobDataFile_,saveLobDataFile);
+ if (rc2 == -1)
+ {
+ lobDebugInfo("Problem renaming datafile to save data file",0,__LINE__,lobTrace_);
+ NADELETEBASIC(saveLobDataFile,getLobGlobalHeap());
+ return LOB_DATA_FILE_WRITE_ERROR;
+ }
+ //create a new file of the same name.
+ hdfsFile fdNew = hdfsOpenFile(fs_, lobDataFile_,O_WRONLY|O_CREAT,0,0,0);
+ if (!fdNew)
+ {
+ str_sprintf(logBuf,"Could not create/open file:%s",lobDataFile_);
+ lobDebugInfo(logBuf,0,__LINE__,lobTrace_);
+
+ //restore previous version
+ Int32 rc2 = hdfsRename(fs_,saveLobDataFile,lobDataFile_);
+ if (rc2 == -1)
+ {
+ lobDebugInfo("Problem restoring datafile . Will need to retry the update",0,__LINE__,lobTrace_);
+ NADELETEBASIC(saveLobDataFile,getLobGlobalHeap());
+ return LOB_DATA_FILE_WRITE_ERROR;
+ }
+ return LOB_DATA_FILE_OPEN_ERROR;
--- End diff --
But we don't reach the "else" part. If at line 1531, rc is not -1, we proceed to line 1537 and do a return. Unless I'm not reading this correctly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94450083
--- Diff: core/sql/cli/Cli.cpp ---
@@ -10121,6 +10122,23 @@ Lng32 SQLCLI_LOBcliInterface
if (inoutDescSyskey)
*inoutDescSyskey = inDescSyskey;
}
+ else
+ {
+ if (cliRC == 100)
+ {
+ if (dataLen)
+ *dataLen = 0;
+ if (dataOffset)
+ *dataOffset = 0;
+ if (blackBoxLen)
+ *blackBoxLen = 0;
+ if (inoutDescPartnKey)
+ *inoutDescPartnKey = descPartnKey;
+
+ if (inoutDescSyskey)
+ *inoutDescSyskey = inDescSyskey;
+ }
+ }
--- End diff --
Should the two commented lines after this be deleted now (since you're handling the cliRC == 100 case)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94468383
--- Diff: core/sql/cli/Cli.cpp ---
@@ -10678,7 +10697,7 @@ Lng32 SQLCLI_LOBddlInterface
case LOB_CLI_CREATE:
{
// create lob metadata table
- str_sprintf(query, "create ghost table %s (lobnum smallint not null, storagetype smallint not null, location varchar(4096) not null, primary key (lobnum)) ",lobMDName);
+ str_sprintf(query, "create ghost table %s (lobnum smallint not null, storagetype smallint not null, location varchar(4096) not null, column_name varchar(256 bytes) character set utf8, primary key (lobnum)) ",lobMDName);
--- End diff --
Not yet since it's not externalized.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94467176
--- Diff: core/sql/cli/Cli.cpp ---
@@ -10678,7 +10697,7 @@ Lng32 SQLCLI_LOBddlInterface
case LOB_CLI_CREATE:
{
// create lob metadata table
- str_sprintf(query, "create ghost table %s (lobnum smallint not null, storagetype smallint not null, location varchar(4096) not null, primary key (lobnum)) ",lobMDName);
+ str_sprintf(query, "create ghost table %s (lobnum smallint not null, storagetype smallint not null, location varchar(4096) not null, column_name varchar(256 bytes) character set utf8, primary key (lobnum)) ",lobMDName);
--- End diff --
Is there any upgrade logic needed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94468296
--- Diff: core/sql/exp/ExpLOBaccess.cpp ---
@@ -1502,7 +1507,50 @@ Ex_Lob_Error ExLob::allocateDesc(ULng32 size, Int64 &descNum, Int64 &dataOffset,
char logBuf[4096];
lobDebugInfo("In ExLob::allocateDesc",0,__LINE__,lobTrace_);
Int32 openFlags = O_RDONLY ;
-
+ if (size == 0) //we are trying to empty this lob.
+ {
+ //rename lob datafile
+ char * saveLobDataFile = new(getLobGlobalHeap()) char[MAX_LOB_FILE_NAME_LEN+6];
+ str_sprintf(saveLobDataFile, "%s_save",lobDataFile_);
+ Int32 rc2 = hdfsRename(fs_,lobDataFile_,saveLobDataFile);
+ if (rc2 == -1)
+ {
+ lobDebugInfo("Problem renaming datafile to save data file",0,__LINE__,lobTrace_);
+ NADELETEBASIC(saveLobDataFile,getLobGlobalHeap());
+ return LOB_DATA_FILE_WRITE_ERROR;
+ }
+ //create a new file of the same name.
+ hdfsFile fdNew = hdfsOpenFile(fs_, lobDataFile_,O_WRONLY|O_CREAT,0,0,0);
+ if (!fdNew)
+ {
+ str_sprintf(logBuf,"Could not create/open file:%s",lobDataFile_);
+ lobDebugInfo(logBuf,0,__LINE__,lobTrace_);
+
+ //restore previous version
+ Int32 rc2 = hdfsRename(fs_,saveLobDataFile,lobDataFile_);
+ if (rc2 == -1)
+ {
+ lobDebugInfo("Problem restoring datafile . Will need to retry the update",0,__LINE__,lobTrace_);
+ NADELETEBASIC(saveLobDataFile,getLobGlobalHeap());
+ return LOB_DATA_FILE_WRITE_ERROR;
+ }
+ return LOB_DATA_FILE_OPEN_ERROR;
--- End diff --
You're right - there was that code path where it was missing. Adding that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94461709
--- Diff: core/sql/exp/ExpLOBaccess.cpp ---
@@ -1502,7 +1507,50 @@ Ex_Lob_Error ExLob::allocateDesc(ULng32 size, Int64 &descNum, Int64 &dataOffset,
char logBuf[4096];
lobDebugInfo("In ExLob::allocateDesc",0,__LINE__,lobTrace_);
Int32 openFlags = O_RDONLY ;
-
+ if (size == 0) //we are trying to empty this lob.
+ {
+ //rename lob datafile
+ char * saveLobDataFile = new(getLobGlobalHeap()) char[MAX_LOB_FILE_NAME_LEN+6];
+ str_sprintf(saveLobDataFile, "%s_save",lobDataFile_);
+ Int32 rc2 = hdfsRename(fs_,lobDataFile_,saveLobDataFile);
+ if (rc2 == -1)
+ {
+ lobDebugInfo("Problem renaming datafile to save data file",0,__LINE__,lobTrace_);
+ NADELETEBASIC(saveLobDataFile,getLobGlobalHeap());
+ return LOB_DATA_FILE_WRITE_ERROR;
+ }
+ //create a new file of the same name.
+ hdfsFile fdNew = hdfsOpenFile(fs_, lobDataFile_,O_WRONLY|O_CREAT,0,0,0);
+ if (!fdNew)
+ {
+ str_sprintf(logBuf,"Could not create/open file:%s",lobDataFile_);
+ lobDebugInfo(logBuf,0,__LINE__,lobTrace_);
+
+ //restore previous version
+ Int32 rc2 = hdfsRename(fs_,saveLobDataFile,lobDataFile_);
+ if (rc2 == -1)
+ {
+ lobDebugInfo("Problem restoring datafile . Will need to retry the update",0,__LINE__,lobTrace_);
+ NADELETEBASIC(saveLobDataFile,getLobGlobalHeap());
+ return LOB_DATA_FILE_WRITE_ERROR;
+ }
+ return LOB_DATA_FILE_OPEN_ERROR;
--- End diff --
its deleted below in the "else" part
else
{
//A new empty data file has been created.
// delete the saved data file
Int32 rc2 = hdfsDelete(fs_,saveLobDataFile,FALSE);//ok to ignore error.nt32
if (rc2 == -1)
{
lobDebugInfo("Problem deleting saved datafile . Will need to manually cleanup saved datafile",0,__LINE__,lobTrace_);
}
NADELETEBASIC(saveLobDataFile,getLobGlobalHeap());
hdfsCloseFile(fs_,fdNew);
fdNew = NULL;
}
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94455073
--- Diff: core/sql/optimizer/SynthType.cpp ---
@@ -6763,19 +6772,28 @@ const NAType *LOBupdate::synthesizeType()
{
// Return blob type
- ValueId vid1 = child(0)->getValueId();
- const NAType &typ1 = (NAType&)vid1.getType();
+ ValueId vid1,vid2 ;
+ const NAType *typ1,*typ2 = NULL;
- ValueId vid2 = child(1)->getValueId();
- const NAType &typ2 = (NAType&)vid2.getType();
+ if(child(0))
+ {
+ vid1= child(0)->getValueId();
+ typ1 = &vid1.getType();
+ }
+
+ if(child(1))
+ {
+ vid2 = child(1)->getValueId();
+ typ2 = &vid2.getType();
+ }
if ((obj_ == STRING_) ||
(obj_ == FILE_) ||
(obj_ == EXTERNAL_))
{
- if (typ1.getTypeQualifier() != NA_CHARACTER_TYPE)
+ if (typ1->getTypeQualifier() != NA_CHARACTER_TYPE)
--- End diff --
Could typ1 be null? Should we check for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94453751
--- Diff: core/sql/exp/ExpLOBaccess.cpp ---
@@ -1502,7 +1507,50 @@ Ex_Lob_Error ExLob::allocateDesc(ULng32 size, Int64 &descNum, Int64 &dataOffset,
char logBuf[4096];
lobDebugInfo("In ExLob::allocateDesc",0,__LINE__,lobTrace_);
Int32 openFlags = O_RDONLY ;
-
+ if (size == 0) //we are trying to empty this lob.
+ {
+ //rename lob datafile
+ char * saveLobDataFile = new(getLobGlobalHeap()) char[MAX_LOB_FILE_NAME_LEN+6];
--- End diff --
Should this be MAX_LOB_FILE_NAME_LEN+6+1 (to allow for "s_save" and for a trailing null)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94451157
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -267,3 +267,107 @@ Int32 updateAppendBufferToLob(CliGlobals *cliglob, char *tableName, char *column
return retcode;
}
+
+
+Int32 updateBufferToLobHandle(CliGlobals *cliglob,char *handle)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // update lob data into via a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
+ str_sprintf(query,"update lob (LOB '%s' , LOCATION %Ld, SIZE %Ld)", handle, (Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ {
+ cliInterface.executeImmediate("rollback work");
+ delete query;
+ delete lobDataBuf;
+ return retcode;
+ }
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+Int32 updateAppendBufferToLobHandle(CliGlobals *cliglob,char *handle)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // update lob data into via a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
+ str_sprintf(query,"update lob (LOB '%s' , LOCATION %Ld, SIZE %Ld,append )", handle, (Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ {
+ cliInterface.executeImmediate("rollback work");
+ delete query;
+ delete lobDataBuf;
+ return retcode;
+ }
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 updateTruncateLobHandle(CliGlobals *cliglob,char *handle)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // update lob data into via a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
--- End diff --
Buffer overrun
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-trafodion/pull/889
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94458926
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -267,3 +267,107 @@ Int32 updateAppendBufferToLob(CliGlobals *cliglob, char *tableName, char *column
return retcode;
}
+
+
+Int32 updateBufferToLobHandle(CliGlobals *cliglob,char *handle)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // update lob data into via a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
--- End diff --
Yes - fixed this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94450518
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -267,3 +267,107 @@ Int32 updateAppendBufferToLob(CliGlobals *cliglob, char *tableName, char *column
return retcode;
}
+
+
+Int32 updateBufferToLobHandle(CliGlobals *cliglob,char *handle)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // update lob data into via a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
--- End diff --
Buffer overrun... lobDataBuf looks like it is only 10 bytes long.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94452427
--- Diff: core/sql/exp/ExpLOB.cpp ---
@@ -1329,7 +1350,60 @@ ex_expr::exp_return_type ExpLOBupdate::eval(char *op_data[],
memcpy(&lobLen, op_data[3],sizeof(Int64)); // user specified buffer length
memcpy(data,op_data[1],sizeof(Int64)); // user buffer address
}
- if (isAppend())
+
+ if(fromEmpty())
+ {
+ lobLen = 0;
+ so = Lob_Memory;
+ }
+ /* if (fromEmpty())
--- End diff --
What's the purpose for the commented-out code?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94466903
--- Diff: core/sql/exp/ExpLOBaccess.cpp ---
@@ -1502,7 +1507,50 @@ Ex_Lob_Error ExLob::allocateDesc(ULng32 size, Int64 &descNum, Int64 &dataOffset,
char logBuf[4096];
lobDebugInfo("In ExLob::allocateDesc",0,__LINE__,lobTrace_);
Int32 openFlags = O_RDONLY ;
-
+ if (size == 0) //we are trying to empty this lob.
+ {
+ //rename lob datafile
+ char * saveLobDataFile = new(getLobGlobalHeap()) char[MAX_LOB_FILE_NAME_LEN+6];
--- End diff --
Ah, yes... my mistake. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94459738
--- Diff: core/sql/executor/ExExeUtil.h ---
@@ -2970,8 +2970,92 @@ class ExExeUtilLobExtractTcb : public ExExeUtilTcb
LOBglobals *lobGlobals_;
};
+
+
+//////////////////////////////////////////////////////////////////////////
+// -----------------------------------------------------------------------
+// ExExeUtilLobUpdateTdb
+// -----------------------------------------------------------------------
+class ExExeUtilLobUpdateTdb : public ComTdbExeUtilLobUpdate
+{
+ public:
+
+ // ---------------------------------------------------------------------
+ // Constructor is only called to instantiate an object used for
+ // retrieval of the virtual table function pointer of the class while
+ // unpacking. An empty constructor is enough.
+ // ---------------------------------------------------------------------
+ NA_EIDPROC ExExeUtilLobUpdateTdb()
+ {}
+
+ NA_EIDPROC virtual ~ExExeUtilLobUpdateTdb()
+ {}
+
+
+ // ---------------------------------------------------------------------
+ // Build a TCB for this TDB. Redefined in the Executor project.
+ // ---------------------------------------------------------------------
+ NA_EIDPROC virtual ex_tcb *build(ex_globals *globals);
+private:
+};
+
+//////////////////////////////////////////////////////////////////////////
+// -----------------------------------------------------------------------
+// ExExeUtilLobUpdateTcb
+// -----------------------------------------------------------------------
+
+
+class ExExeUtilLobUpdateTcb : public ExExeUtilTcb
+{
+ friend class ExExeUtilLobUpdateTdb;
+ friend class ExExeUtilPrivateState;
+
+public:
+ // Constructor
+ ExExeUtilLobUpdateTcb(const ComTdbExeUtilLobUpdate & exe_util_tdb,
+ const ex_tcb * child_tcb,
+ ex_globals * glob = 0);
+
+ virtual short work();
+
+ ExExeUtilLobUpdateTdb & lobTdb() const
+ {
+ return (ExExeUtilLobUpdateTdb &) tdb;
+ };
+ LOBglobals *getLobGlobals() { return lobGlobals_;}
+ protected:
+ enum Step
+ {
+ EMPTY_,
+ GET_HANDLE_,
+ UPDATE_LOB_DATA_,
+ EMPTY_LOB_DATA_,
+ APPEND_LOB_DATA_,
+ RETURN_STATUS_,
+ DONE_,
+ CANCEL_,
+ HANDLE_ERROR_,
+ CANCELLED_
+ };
+ Step step_;
+ Lng32 lobHandleLen_;
+ char lobHandle_[2050];
--- End diff --
Cloned from the ExtractTcb class code above this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94451970
--- Diff: core/sql/executor/ExExeUtil.h ---
@@ -2970,8 +2970,92 @@ class ExExeUtilLobExtractTcb : public ExExeUtilTcb
LOBglobals *lobGlobals_;
};
+
+
+//////////////////////////////////////////////////////////////////////////
+// -----------------------------------------------------------------------
+// ExExeUtilLobUpdateTdb
+// -----------------------------------------------------------------------
+class ExExeUtilLobUpdateTdb : public ComTdbExeUtilLobUpdate
+{
+ public:
+
+ // ---------------------------------------------------------------------
+ // Constructor is only called to instantiate an object used for
+ // retrieval of the virtual table function pointer of the class while
+ // unpacking. An empty constructor is enough.
+ // ---------------------------------------------------------------------
+ NA_EIDPROC ExExeUtilLobUpdateTdb()
+ {}
+
+ NA_EIDPROC virtual ~ExExeUtilLobUpdateTdb()
+ {}
+
+
+ // ---------------------------------------------------------------------
+ // Build a TCB for this TDB. Redefined in the Executor project.
+ // ---------------------------------------------------------------------
+ NA_EIDPROC virtual ex_tcb *build(ex_globals *globals);
+private:
+};
+
+//////////////////////////////////////////////////////////////////////////
+// -----------------------------------------------------------------------
+// ExExeUtilLobUpdateTcb
+// -----------------------------------------------------------------------
+
+
+class ExExeUtilLobUpdateTcb : public ExExeUtilTcb
+{
+ friend class ExExeUtilLobUpdateTdb;
+ friend class ExExeUtilPrivateState;
+
+public:
+ // Constructor
+ ExExeUtilLobUpdateTcb(const ComTdbExeUtilLobUpdate & exe_util_tdb,
+ const ex_tcb * child_tcb,
+ ex_globals * glob = 0);
+
+ virtual short work();
+
+ ExExeUtilLobUpdateTdb & lobTdb() const
+ {
+ return (ExExeUtilLobUpdateTdb &) tdb;
+ };
+ LOBglobals *getLobGlobals() { return lobGlobals_;}
+ protected:
+ enum Step
+ {
+ EMPTY_,
+ GET_HANDLE_,
+ UPDATE_LOB_DATA_,
+ EMPTY_LOB_DATA_,
+ APPEND_LOB_DATA_,
+ RETURN_STATUS_,
+ DONE_,
+ CANCEL_,
+ HANDLE_ERROR_,
+ CANCELLED_
+ };
+ Step step_;
+ Lng32 lobHandleLen_;
+ char lobHandle_[2050];
--- End diff --
How are the lengths of these variables derived?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request #889: [TRAFODION-2408] Changes to support "...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/889#discussion_r94450683
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -267,3 +267,107 @@ Int32 updateAppendBufferToLob(CliGlobals *cliglob, char *tableName, char *column
return retcode;
}
+
+
+Int32 updateBufferToLobHandle(CliGlobals *cliglob,char *handle)
--- End diff --
What's the expected maximum length of handle?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---