You are viewing a plain text version of this content. The canonical link for it is here.
Posted to codereview@trafodion.apache.org by sandhyasun <gi...@git.apache.org> on 2016/01/05 22:46:07 UTC
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
GitHub user sandhyasun opened a pull request:
https://github.com/apache/incubator-trafodion/pull/248
Support for extracting lob to buffer and buffertolob() function .JIRA : TRAFODION-1473
*Support has been added to support new syntax that can extract to a user buffer.(Support for extracting into a file already there)
The new syntax looks as follows :
extract from <table> (lob <lobhandle>, LOCATION <user buffer address as long>, SIZE <user buffer address as long>
Having the SIZE as an input/output allows the user to retrieve how much was actually extracted.
For eg they could input a buffer of 1000 and SIZE 1000 but get back only 750 bytes of data.
Although this syntax is going to be used mainly by jdbc applications and programs, to test it via SQL interfaces (for developers only) a test driver has been added which gets built as part of the build . The regression test executor/TEST130 has been enhanced to call the testdriver to test this interface.
*Support has been added to support reading from a user buffer into a lob.(Support for reading from a string in memory or a file is already there)
*Syntax :
insert into <table> values (...,buffertolob(LOCATION <user buffer address>, SIZE < sizeaddr>)
update <table> set <lob column> = buffertolob(LOCATION <user buffer address>, SIZE <sizeaddr>)
*Added new Ansi datatypes -602 (SQLTYPECODE_BLOB) and -603(SQLTYPECODE_CLOB)
*Added a clitestdriver( for dev use only) for easy testing and development of new interfaces which could other be tested only via jdbc programs.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sandhyasun/incubator-trafodion lob_work_buffer
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/248.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #248
----
commit dd732847da86e2971766c5b169b0663093320e29
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2015-12-09T07:26:42Z
Suppport for extractingl lob data to a user buffer inn a cursor-like fashion.
Added a cli test interface to programmatically dynamic queries such as lob extract.
commit 5cfc9fa4e70eaeb041a03499fb2e7dd2fae7d28e
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2015-12-09T22:39:20Z
Recommitting the last commit. Some files did notget added. Context/changes are the same as earlier commit.
commit 353e1f3b8d619e5c5ee4934d6b58a6bf2b7aa542
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2015-12-09T22:42:23Z
sqvers for clitestdriver
commit 781682afe4ee662e06e9e9da6b0753e4f3d9d673
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2015-12-11T19:25:39Z
Changed output param for SIZE to be an input/output address.
commit e4f391f3dff713cf5b24c1e3860cfdc220a3a557
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2015-12-17T05:29:49Z
Regression test for testing the programmatic interface for extracting lob data into buffer like a cursor.
commit 2ba37e8e1091f210a2640eef4ea614837a7f6f09
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2015-12-18T22:40:55Z
Merge remote branch 'origin/master' into lob_work_buffer
Conflicts:
core/sql/regress/executor/EXPECTED130
core/sql/regress/executor/TEST130
commit 96ac8ebdb8b7df9f4da42352511a1c4411ada2bb
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2015-12-21T20:47:30Z
More adjustments to lob regress test
commit 3fcf18813fd9f7b5ecce041b2675548e82f7e574
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2016-01-04T18:36:25Z
insert/update LOB support from external user buffer
commit fd80e81f0435c091ee7f1e6a7bce320028abed03
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2016-01-05T18:04:41Z
Additional fixes to deal with new Ansi blob datatype.
commit 13f897e8f6600138fd4a73594847a40faa328e40
Author: Sandhya Sundaresan <sa...@apache.org>
Date: 2016-01-05T21:42:39Z
added copyrights to new testdriver files
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979535
--- Diff: core/sql/nskgmake/clitest/Makefile~ ---
@@ -0,0 +1,35 @@
+#######################################################################
--- End diff --
Not sure how this got committed by mistake. Removed it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979545
--- Diff: core/sql/nskgmake/clitestdriver/Makefile~ ---
@@ -0,0 +1,35 @@
+#######################################################################
--- End diff --
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979502
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -2890,7 +2890,12 @@ short ExExeUtilLobExtractTcb::work()
}
- step_ = RETRIEVE_LOB_LENGTH_;
+ if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_BUFFER_)
+ step_ = EXTRACT_LOB_DATA_;
+ else
+ if ((lobTdb().getToType() == ComTdbExeUtilLobExtract::RETRIEVE_LENGTH_) || (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_))
+ step_ = RETRIEVE_LOB_LENGTH_;
+
--- End diff --
Added a block to handle an invalid "toType" to get out of any invalid state .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911708
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**********************************************************************
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**********************************************************************/
+/* -*-C++-*-
+****************************************************************************
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description: Test driver useing exe util cli interface
+*
+*
+*
+*
+****************************************************************************
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle,
+ Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ char * query = new char[4096];
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ //Use lob handle to retrieve the lob length.
+ char lobLengthResult[200];
+ str_cpy_all(lobLengthResult," ",200);
+ Int32 lobLengthResultLen = 0;
+ str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld ",lobHandle, &lengthOfLob);
+ retcode = cliInterface.executeImmediate(query,lobLengthResult,&lobLengthResultLen,FALSE);
+
+ delete query;
+ return retcode;
+
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ char * query = new char[4096];
+ Int32 lobHandleLen = 0;
+ str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+
+ retcode = cliInterface.executeImmediate(query,lobHandle,&lobHandleLen,FALSE);
+
+ if (retcode)
+ return retcode;
+ lobHandle[lobHandleLen]='\0';
+ delete query;
+
+ return retcode;
+
+}
+
+Int32 extractLobToBuffer(CliGlobals *cliglob, char * lobHandle, Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+ char *lobFinalBuf = new char[lengthOfLob];
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, &lobExtractLen);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ memcpy((char*)&(lobFinalBuf[i]),(char *)lobDataBuf,lobExtractLen);
+ i += lobExtractLen;
+ }
+ }
+ if (retcode ==100 || retcode ==0)
+ {
+ FILE * lobFileId = fopen("lob_output_file","w");
+
+ int byteCount=fwrite(lobFinalBuf,sizeof(char),lengthOfLob, lobFileId);
+ cout << "Writing " << byteCount << " bytes from user buffer to file lob_output_file in current directory" << endl;
+
+ fclose(lobFileId);
+ }
+ delete lobFinalBuf;
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 extractLobToFileInChunks(CliGlobals *cliglob, char * lobHandle, char *filename,Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+ Int64 *inputOutputAddr = &lobExtractLen;
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, inputOutputAddr);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ FILE * lobFileId = fopen(filename,"a+");
+ int byteCount = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ byteCount=fwrite(lobDataBuf,sizeof(char),*inputOutputAddr, lobFileId);
+ cout << "Wrote " << byteCount << " bytes to file : " << filename << endl;
+ }
+ }
+
+
+ fclose(lobFileId);
+
+
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 insertBufferToLob(CliGlobals *cliglob, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobInsertLen = 10;
+ char *lobDataBuf = new char[lobInsertLen];
+ memcpy(lobDataBuf, "xxxxxyyyyy",10);
+ str_sprintf(query,"insert into %s values (1, buffertolob (LOCATION %Ld, SIZE %Ld))", tableName,(Int64)lobDataBuf, lobInsertLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
--- End diff --
Leaks the heap variable memory
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912365
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -3098,19 +3119,25 @@ short ExExeUtilLobExtractTcb::work()
break;
}
- if (lobDataLen_ == 0)
+ if (lobDataOutputLen == 0)
{
step_ = CLOSE_CURSOR_;
break;
}
- remainingBytes_ = (Lng32)lobDataLen_;
+ remainingBytes_ = (Lng32)lobDataOutputLen;
currPos_ = 0;
- if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
- step_ = INSERT_FROM_STRING_;
- else
- step_ = RETURN_STRING_;
+ /*if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
+ step_ = INSERT_FROM_STRING_;*/
+ if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_BUFFER_)
+ {
+ str_sprintf(statusString_," Success: LOB data length returned : %d", lobDataOutputLen);
+
+ //lobTdb().setExtractSizeIOAddr((Int64)(&lobDataOutputLen));
+ memcpy((char *)lobTdb().dataExtractSizeIOAddr(), (char *)&lobDataOutputLen,sizeof(Int64));
+ step_ = RETURN_STATUS_;
+ }
--- End diff --
Else we remain in the same step_? Could that cause an infinite loop?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979468
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**********************************************************************
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**********************************************************************/
+/* -*-C++-*-
+****************************************************************************
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description: Test driver useing exe util cli interface
+*
+*
+*
+*
+****************************************************************************
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle,
+ Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ char * query = new char[4096];
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ //Use lob handle to retrieve the lob length.
+ char lobLengthResult[200];
+ str_cpy_all(lobLengthResult," ",200);
+ Int32 lobLengthResultLen = 0;
+ str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld ",lobHandle, &lengthOfLob);
+ retcode = cliInterface.executeImmediate(query,lobLengthResult,&lobLengthResultLen,FALSE);
+
+ delete query;
+ return retcode;
+
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ char * query = new char[4096];
+ Int32 lobHandleLen = 0;
+ str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+
+ retcode = cliInterface.executeImmediate(query,lobHandle,&lobHandleLen,FALSE);
+
+ if (retcode)
+ return retcode;
+ lobHandle[lobHandleLen]='\0';
+ delete query;
+
+ return retcode;
+
+}
+
+Int32 extractLobToBuffer(CliGlobals *cliglob, char * lobHandle, Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+ char *lobFinalBuf = new char[lengthOfLob];
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, &lobExtractLen);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ memcpy((char*)&(lobFinalBuf[i]),(char *)lobDataBuf,lobExtractLen);
+ i += lobExtractLen;
+ }
+ }
+ if (retcode ==100 || retcode ==0)
+ {
+ FILE * lobFileId = fopen("lob_output_file","w");
+
+ int byteCount=fwrite(lobFinalBuf,sizeof(char),lengthOfLob, lobFileId);
+ cout << "Writing " << byteCount << " bytes from user buffer to file lob_output_file in current directory" << endl;
+
+ fclose(lobFileId);
+ }
+ delete lobFinalBuf;
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 extractLobToFileInChunks(CliGlobals *cliglob, char * lobHandle, char *filename,Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+ Int64 *inputOutputAddr = &lobExtractLen;
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, inputOutputAddr);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ FILE * lobFileId = fopen(filename,"a+");
+ int byteCount = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ byteCount=fwrite(lobDataBuf,sizeof(char),*inputOutputAddr, lobFileId);
+ cout << "Wrote " << byteCount << " bytes to file : " << filename << endl;
+ }
+ }
+
+
+ fclose(lobFileId);
+
+
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 insertBufferToLob(CliGlobals *cliglob, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobInsertLen = 10;
+ char *lobDataBuf = new char[lobInsertLen];
+ memcpy(lobDataBuf, "xxxxxyyyyy",10);
+ str_sprintf(query,"insert into %s values (1, buffertolob (LOCATION %Ld, SIZE %Ld))", tableName,(Int64)lobDataBuf, lobInsertLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 updateBufferToLob(CliGlobals *cliglob, char *tableName, char *columnName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
+ str_sprintf(query,"update %s set %s= buffertolob(LOCATION %Ld, SIZE %Ld)", tableName,columnName, (Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
--- End diff --
Fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912122
--- Diff: core/sql/executor/ExExeUtilCli.cpp ---
@@ -1048,6 +1052,77 @@ Lng32 ExeCliInterface::executeImmediateExec(const char * stmtStr,
return ((cliRetcode != -1) ? cliRetcode : retcode);
}
+
+Lng32 ExeCliInterface::executeImmediateExecNoDealloc(const char * stmtStr,
+ char * outputBuf,
+ Lng32 * outputBufLen,
+ NABoolean nullTerminate,
+ Int64 * rowsAffected
+ )
+{
+ Lng32 retcode = 0;
+
+ retcode = exec();
+ if (retcode < 0)
+ {
+ deallocStuff(module_, stmt_, sql_src_, input_desc_, output_desc_);
+ return retcode;
+ }
+
+ retcode = fetch();
+ if (retcode < 0)
+ {
+ deallocStuff(module_, stmt_, sql_src_, input_desc_, output_desc_);
+ return retcode;
+ }
+
+ if ((outputBuf) &&
+ (outputBufLen))
+ {
+ *outputBufLen = 0;
+ if (retcode != 100)
+ {
+ char * ptr;
+ Lng32 len;
+ getPtrAndLen(1, ptr, len);
+
+ str_cpy_all(outputBuf, ptr, len);
--- End diff --
Is there any buffer overrun checking done anywhere?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911725
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**********************************************************************
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**********************************************************************/
+/* -*-C++-*-
+****************************************************************************
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description: Test driver useing exe util cli interface
+*
+*
+*
+*
+****************************************************************************
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle,
+ Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ char * query = new char[4096];
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ //Use lob handle to retrieve the lob length.
+ char lobLengthResult[200];
+ str_cpy_all(lobLengthResult," ",200);
+ Int32 lobLengthResultLen = 0;
+ str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld ",lobHandle, &lengthOfLob);
+ retcode = cliInterface.executeImmediate(query,lobLengthResult,&lobLengthResultLen,FALSE);
+
+ delete query;
+ return retcode;
+
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ char * query = new char[4096];
+ Int32 lobHandleLen = 0;
+ str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+
+ retcode = cliInterface.executeImmediate(query,lobHandle,&lobHandleLen,FALSE);
+
+ if (retcode)
+ return retcode;
+ lobHandle[lobHandleLen]='\0';
+ delete query;
+
+ return retcode;
+
+}
+
+Int32 extractLobToBuffer(CliGlobals *cliglob, char * lobHandle, Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+ char *lobFinalBuf = new char[lengthOfLob];
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, &lobExtractLen);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ memcpy((char*)&(lobFinalBuf[i]),(char *)lobDataBuf,lobExtractLen);
+ i += lobExtractLen;
+ }
+ }
+ if (retcode ==100 || retcode ==0)
+ {
+ FILE * lobFileId = fopen("lob_output_file","w");
+
+ int byteCount=fwrite(lobFinalBuf,sizeof(char),lengthOfLob, lobFileId);
+ cout << "Writing " << byteCount << " bytes from user buffer to file lob_output_file in current directory" << endl;
+
+ fclose(lobFileId);
+ }
+ delete lobFinalBuf;
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 extractLobToFileInChunks(CliGlobals *cliglob, char * lobHandle, char *filename,Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+ Int64 *inputOutputAddr = &lobExtractLen;
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, inputOutputAddr);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ FILE * lobFileId = fopen(filename,"a+");
+ int byteCount = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ byteCount=fwrite(lobDataBuf,sizeof(char),*inputOutputAddr, lobFileId);
+ cout << "Wrote " << byteCount << " bytes to file : " << filename << endl;
+ }
+ }
+
+
+ fclose(lobFileId);
+
+
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 insertBufferToLob(CliGlobals *cliglob, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobInsertLen = 10;
+ char *lobDataBuf = new char[lobInsertLen];
+ memcpy(lobDataBuf, "xxxxxyyyyy",10);
+ str_sprintf(query,"insert into %s values (1, buffertolob (LOCATION %Ld, SIZE %Ld))", tableName,(Int64)lobDataBuf, lobInsertLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 updateBufferToLob(CliGlobals *cliglob, char *tableName, char *columnName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
+ str_sprintf(query,"update %s set %s= buffertolob(LOCATION %Ld, SIZE %Ld)", tableName,columnName, (Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
--- End diff --
Memory leak
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979519
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -3098,19 +3119,25 @@ short ExExeUtilLobExtractTcb::work()
break;
}
- if (lobDataLen_ == 0)
+ if (lobDataOutputLen == 0)
{
step_ = CLOSE_CURSOR_;
break;
}
- remainingBytes_ = (Lng32)lobDataLen_;
+ remainingBytes_ = (Lng32)lobDataOutputLen;
currPos_ = 0;
- if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
- step_ = INSERT_FROM_STRING_;
- else
- step_ = RETURN_STRING_;
+ /*if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
+ step_ = INSERT_FROM_STRING_;*/
--- End diff --
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912575
--- Diff: core/sql/nskgmake/clitest/Makefile~ ---
@@ -0,0 +1,35 @@
+#######################################################################
--- End diff --
I'm guessing you did not mean to add this file (Makefile~) to your commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979472
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**********************************************************************
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**********************************************************************/
+/* -*-C++-*-
+****************************************************************************
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description: Test driver useing exe util cli interface
+*
+*
+*
+*
+****************************************************************************
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle,
+ Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ char * query = new char[4096];
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ //Use lob handle to retrieve the lob length.
+ char lobLengthResult[200];
+ str_cpy_all(lobLengthResult," ",200);
+ Int32 lobLengthResultLen = 0;
+ str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld ",lobHandle, &lengthOfLob);
+ retcode = cliInterface.executeImmediate(query,lobLengthResult,&lobLengthResultLen,FALSE);
+
+ delete query;
+ return retcode;
+
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ char * query = new char[4096];
+ Int32 lobHandleLen = 0;
+ str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+
+ retcode = cliInterface.executeImmediate(query,lobHandle,&lobHandleLen,FALSE);
+
+ if (retcode)
+ return retcode;
+ lobHandle[lobHandleLen]='\0';
+ delete query;
+
+ return retcode;
+
+}
+
+Int32 extractLobToBuffer(CliGlobals *cliglob, char * lobHandle, Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+ char *lobFinalBuf = new char[lengthOfLob];
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, &lobExtractLen);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ memcpy((char*)&(lobFinalBuf[i]),(char *)lobDataBuf,lobExtractLen);
+ i += lobExtractLen;
+ }
+ }
+ if (retcode ==100 || retcode ==0)
+ {
+ FILE * lobFileId = fopen("lob_output_file","w");
+
+ int byteCount=fwrite(lobFinalBuf,sizeof(char),lengthOfLob, lobFileId);
+ cout << "Writing " << byteCount << " bytes from user buffer to file lob_output_file in current directory" << endl;
+
+ fclose(lobFileId);
+ }
+ delete lobFinalBuf;
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 extractLobToFileInChunks(CliGlobals *cliglob, char * lobHandle, char *filename,Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+ Int64 *inputOutputAddr = &lobExtractLen;
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, inputOutputAddr);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ FILE * lobFileId = fopen(filename,"a+");
+ int byteCount = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ byteCount=fwrite(lobDataBuf,sizeof(char),*inputOutputAddr, lobFileId);
+ cout << "Wrote " << byteCount << " bytes to file : " << filename << endl;
+ }
+ }
+
+
+ fclose(lobFileId);
+
+
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 insertBufferToLob(CliGlobals *cliglob, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobInsertLen = 10;
+ char *lobDataBuf = new char[lobInsertLen];
+ memcpy(lobDataBuf, "xxxxxyyyyy",10);
+ str_sprintf(query,"insert into %s values (1, buffertolob (LOCATION %Ld, SIZE %Ld))", tableName,(Int64)lobDataBuf, lobInsertLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 updateBufferToLob(CliGlobals *cliglob, char *tableName, char *columnName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
+ str_sprintf(query,"update %s set %s= buffertolob(LOCATION %Ld, SIZE %Ld)", tableName,columnName, (Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+
+Int32 updateAppendBufferToLob(CliGlobals *cliglob, char *tableName, char *columnName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "aaaaabbbbbccccc",15);
+ str_sprintf(query,"update %s set %s=buffertolob (LOCATION %Ld, SIZE %Ld,append)", tableName, columnName,(Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
--- End diff --
Fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979511
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -3098,19 +3119,25 @@ short ExExeUtilLobExtractTcb::work()
break;
}
- if (lobDataLen_ == 0)
+ if (lobDataOutputLen == 0)
{
step_ = CLOSE_CURSOR_;
break;
}
- remainingBytes_ = (Lng32)lobDataLen_;
+ remainingBytes_ = (Lng32)lobDataOutputLen;
currPos_ = 0;
- if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
- step_ = INSERT_FROM_STRING_;
- else
- step_ = RETURN_STRING_;
+ /*if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
+ step_ = INSERT_FROM_STRING_;*/
+ if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_BUFFER_)
+ {
+ str_sprintf(statusString_," Success: LOB data length returned : %d", lobDataOutputLen);
+
+ //lobTdb().setExtractSizeIOAddr((Int64)(&lobDataOutputLen));
+ memcpy((char *)lobTdb().dataExtractSizeIOAddr(), (char *)&lobDataOutputLen,sizeof(Int64));
+ step_ = RETURN_STATUS_;
+ }
--- End diff --
Added an "invalid state handling" block to avoid hanging in the same state
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912261
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -2890,7 +2890,12 @@ short ExExeUtilLobExtractTcb::work()
}
- step_ = RETRIEVE_LOB_LENGTH_;
+ if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_BUFFER_)
+ step_ = EXTRACT_LOB_DATA_;
+ else
+ if ((lobTdb().getToType() == ComTdbExeUtilLobExtract::RETRIEVE_LENGTH_) || (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_))
+ step_ = RETRIEVE_LOB_LENGTH_;
+
--- End diff --
So, if neither of these 'ifs' are true, do we just remain in the same step_? Could that be an infinite loop?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911736
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**********************************************************************
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**********************************************************************/
+/* -*-C++-*-
+****************************************************************************
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description: Test driver useing exe util cli interface
+*
+*
+*
+*
+****************************************************************************
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle,
+ Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ char * query = new char[4096];
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ //Use lob handle to retrieve the lob length.
+ char lobLengthResult[200];
+ str_cpy_all(lobLengthResult," ",200);
+ Int32 lobLengthResultLen = 0;
+ str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld ",lobHandle, &lengthOfLob);
+ retcode = cliInterface.executeImmediate(query,lobLengthResult,&lobLengthResultLen,FALSE);
+
+ delete query;
+ return retcode;
+
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ char * query = new char[4096];
+ Int32 lobHandleLen = 0;
+ str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+
+ retcode = cliInterface.executeImmediate(query,lobHandle,&lobHandleLen,FALSE);
+
+ if (retcode)
+ return retcode;
+ lobHandle[lobHandleLen]='\0';
+ delete query;
+
+ return retcode;
+
+}
+
+Int32 extractLobToBuffer(CliGlobals *cliglob, char * lobHandle, Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+ char *lobFinalBuf = new char[lengthOfLob];
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, &lobExtractLen);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ memcpy((char*)&(lobFinalBuf[i]),(char *)lobDataBuf,lobExtractLen);
+ i += lobExtractLen;
+ }
+ }
+ if (retcode ==100 || retcode ==0)
+ {
+ FILE * lobFileId = fopen("lob_output_file","w");
+
+ int byteCount=fwrite(lobFinalBuf,sizeof(char),lengthOfLob, lobFileId);
+ cout << "Writing " << byteCount << " bytes from user buffer to file lob_output_file in current directory" << endl;
+
+ fclose(lobFileId);
+ }
+ delete lobFinalBuf;
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 extractLobToFileInChunks(CliGlobals *cliglob, char * lobHandle, char *filename,Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+ Int64 *inputOutputAddr = &lobExtractLen;
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, inputOutputAddr);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ FILE * lobFileId = fopen(filename,"a+");
+ int byteCount = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ byteCount=fwrite(lobDataBuf,sizeof(char),*inputOutputAddr, lobFileId);
+ cout << "Wrote " << byteCount << " bytes to file : " << filename << endl;
+ }
+ }
+
+
+ fclose(lobFileId);
+
+
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 insertBufferToLob(CliGlobals *cliglob, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobInsertLen = 10;
+ char *lobDataBuf = new char[lobInsertLen];
+ memcpy(lobDataBuf, "xxxxxyyyyy",10);
+ str_sprintf(query,"insert into %s values (1, buffertolob (LOCATION %Ld, SIZE %Ld))", tableName,(Int64)lobDataBuf, lobInsertLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 updateBufferToLob(CliGlobals *cliglob, char *tableName, char *columnName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "zzzzzzzzzzzzzzzzzzzz",20);
+ str_sprintf(query,"update %s set %s= buffertolob(LOCATION %Ld, SIZE %Ld)", tableName,columnName, (Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
+
+ retcode = cliInterface.executeImmediate("commit work");
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+
+Int32 updateAppendBufferToLob(CliGlobals *cliglob, char *tableName, char *columnName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobUpdateLen = 10;
+ char *lobDataBuf = new char[lobUpdateLen];
+ memcpy(lobDataBuf, "aaaaabbbbbccccc",15);
+ str_sprintf(query,"update %s set %s=buffertolob (LOCATION %Ld, SIZE %Ld,append)", tableName, columnName,(Int64)lobDataBuf, lobUpdateLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
--- End diff --
Memory leak
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979526
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -3452,12 +3492,12 @@ short ExExeUtilFileExtractTcb::work()
break;
- case RETURN_STRING_:
+ /*case RETURN_STRING_:
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912409
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -3452,12 +3492,12 @@ short ExExeUtilFileExtractTcb::work()
break;
- case RETURN_STRING_:
+ /*case RETURN_STRING_:
--- End diff --
Should the commented-out code be deleted?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911867
--- Diff: core/sql/comexe/ComTdbExeUtil.cpp ---
@@ -2231,28 +2233,32 @@ ComTdbExeUtilLobExtract::ComTdbExeUtilLobExtract
handle_(handle),
handleLen_(handleLen),
toType_((short)toType),
+ bufAddr_(bufAddr),
+ extractSizeIOAddr_(extractSizeAddr),
lobStorageType_(lobStorageType),
stringParam1_(stringParam1),
stringParam2_(stringParam2),
stringParam3_(stringParam3),
lobHdfsServer_(lobHdfsServer),
lobHdfsPort_(lobHdfsPort),
- rowSize_(0),
- bufSize_(0),
+ totalBufSize_(0),
flags_(0)
{
setNodeType(ComTdb::ex_LOB_EXTRACT);
if ((toType_ == ExtractToType::TO_BUFFER_) || (toType_ == ExtractToType::TO_STRING_))
{
- // intparam1 contains the rowsize passed in via syntax
+ /*// intparam1 contains the rowsize passed in via syntax
// intparam2 constains the total buf size user has allocated
rowSize_ = intParam1;
- bufSize_ = intParam2;
+ bufSize_ = intParam2; */
--- End diff --
Should we just delete this block of code?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979493
--- Diff: core/sql/executor/ExExeUtilCli.cpp ---
@@ -1048,6 +1052,77 @@ Lng32 ExeCliInterface::executeImmediateExec(const char * stmtStr,
return ((cliRetcode != -1) ? cliRetcode : retcode);
}
+
+Lng32 ExeCliInterface::executeImmediateExecNoDealloc(const char * stmtStr,
+ char * outputBuf,
+ Lng32 * outputBufLen,
+ NABoolean nullTerminate,
+ Int64 * rowsAffected
+ )
+{
+ Lng32 retcode = 0;
+
+ retcode = exec();
+ if (retcode < 0)
+ {
+ deallocStuff(module_, stmt_, sql_src_, input_desc_, output_desc_);
+ return retcode;
+ }
+
+ retcode = fetch();
+ if (retcode < 0)
+ {
+ deallocStuff(module_, stmt_, sql_src_, input_desc_, output_desc_);
+ return retcode;
+ }
+
+ if ((outputBuf) &&
+ (outputBufLen))
+ {
+ *outputBufLen = 0;
+ if (retcode != 100)
+ {
+ char * ptr;
+ Lng32 len;
+ getPtrAndLen(1, ptr, len);
+
+ str_cpy_all(outputBuf, ptr, len);
--- End diff --
executeImmediateExeNoDealloc is not needed. Had added it when I encountered a bug with cursor access and then forgot to remove . The whole method has been removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911826
--- Diff: core/sql/comexe/ComTdbExeUtil.cpp ---
@@ -2231,28 +2233,32 @@ ComTdbExeUtilLobExtract::ComTdbExeUtilLobExtract
handle_(handle),
handleLen_(handleLen),
toType_((short)toType),
+ bufAddr_(bufAddr),
+ extractSizeIOAddr_(extractSizeAddr),
lobStorageType_(lobStorageType),
stringParam1_(stringParam1),
stringParam2_(stringParam2),
stringParam3_(stringParam3),
lobHdfsServer_(lobHdfsServer),
lobHdfsPort_(lobHdfsPort),
- rowSize_(0),
- bufSize_(0),
+ totalBufSize_(0),
flags_(0)
{
setNodeType(ComTdb::ex_LOB_EXTRACT);
if ((toType_ == ExtractToType::TO_BUFFER_) || (toType_ == ExtractToType::TO_STRING_))
{
- // intparam1 contains the rowsize passed in via syntax
+ /*// intparam1 contains the rowsize passed in via syntax
--- End diff --
Typo?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912379
--- Diff: core/sql/executor/ExExeUtilLoad.cpp ---
@@ -3098,19 +3119,25 @@ short ExExeUtilLobExtractTcb::work()
break;
}
- if (lobDataLen_ == 0)
+ if (lobDataOutputLen == 0)
{
step_ = CLOSE_CURSOR_;
break;
}
- remainingBytes_ = (Lng32)lobDataLen_;
+ remainingBytes_ = (Lng32)lobDataOutputLen;
currPos_ = 0;
- if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
- step_ = INSERT_FROM_STRING_;
- else
- step_ = RETURN_STRING_;
+ /*if (lobTdb().getToType() == ComTdbExeUtilLobExtract::TO_FILE_)
+ step_ = INSERT_FROM_STRING_;*/
--- End diff --
Should the commented-out code be deleted?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48912603
--- Diff: core/sql/nskgmake/clitestdriver/Makefile~ ---
@@ -0,0 +1,35 @@
+#######################################################################
--- End diff --
I'm guessing you did not intend to add this file (Makefile~) to your commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979457
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**********************************************************************
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**********************************************************************/
+/* -*-C++-*-
+****************************************************************************
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description: Test driver useing exe util cli interface
+*
+*
+*
+*
+****************************************************************************
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle,
+ Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ char * query = new char[4096];
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ //Use lob handle to retrieve the lob length.
+ char lobLengthResult[200];
+ str_cpy_all(lobLengthResult," ",200);
+ Int32 lobLengthResultLen = 0;
+ str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld ",lobHandle, &lengthOfLob);
+ retcode = cliInterface.executeImmediate(query,lobLengthResult,&lobLengthResultLen,FALSE);
+
+ delete query;
+ return retcode;
+
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ char * query = new char[4096];
+ Int32 lobHandleLen = 0;
+ str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+
+ retcode = cliInterface.executeImmediate(query,lobHandle,&lobHandleLen,FALSE);
+
+ if (retcode)
+ return retcode;
+ lobHandle[lobHandleLen]='\0';
+ delete query;
+
+ return retcode;
+
+}
+
+Int32 extractLobToBuffer(CliGlobals *cliglob, char * lobHandle, Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+ char *lobFinalBuf = new char[lengthOfLob];
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, &lobExtractLen);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ memcpy((char*)&(lobFinalBuf[i]),(char *)lobDataBuf,lobExtractLen);
+ i += lobExtractLen;
+ }
+ }
+ if (retcode ==100 || retcode ==0)
+ {
+ FILE * lobFileId = fopen("lob_output_file","w");
+
+ int byteCount=fwrite(lobFinalBuf,sizeof(char),lengthOfLob, lobFileId);
+ cout << "Writing " << byteCount << " bytes from user buffer to file lob_output_file in current directory" << endl;
+
+ fclose(lobFileId);
+ }
+ delete lobFinalBuf;
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 extractLobToFileInChunks(CliGlobals *cliglob, char * lobHandle, char *filename,Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobExtractLen = 1000;
+ char *lobDataBuf = new char[lobExtractLen];
+ Int64 *inputOutputAddr = &lobExtractLen;
+
+ str_sprintf(query,"extract lobtobuffer(lob '%s', LOCATION %Ld, SIZE %Ld) ", lobHandle, (Int64)lobDataBuf, inputOutputAddr);
+
+
+ retcode = cliInterface.executeImmediatePrepare(query);
+ short i = 0;
+ FILE * lobFileId = fopen(filename,"a+");
+ int byteCount = 0;
+ while ((retcode != 100) && !(retcode<0))
+ {
+ retcode = cliInterface.clearExecFetchClose(NULL,NULL,statusBuf, &statusBufLen);
+ if (!retcode)
+ {
+ byteCount=fwrite(lobDataBuf,sizeof(char),*inputOutputAddr, lobFileId);
+ cout << "Wrote " << byteCount << " bytes to file : " << filename << endl;
+ }
+ }
+
+
+ fclose(lobFileId);
+
+
+ delete query;
+ delete lobDataBuf;
+
+
+ return retcode;
+
+}
+
+
+Int32 insertBufferToLob(CliGlobals *cliglob, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ // Extract lob data into a buffer.
+ char * query = new char [500];
+
+
+ char statusBuf[200] = {'\0'};
+ Int32 statusBufLen = 0;
+ Int64 lobInsertLen = 10;
+ char *lobDataBuf = new char[lobInsertLen];
+ memcpy(lobDataBuf, "xxxxxyyyyy",10);
+ str_sprintf(query,"insert into %s values (1, buffertolob (LOCATION %Ld, SIZE %Ld))", tableName,(Int64)lobDataBuf, lobInsertLen);
+
+
+ retcode = cliInterface.executeImmediate(query);
+ if (retcode <0)
+ return retcode;
--- End diff --
Fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979483
--- Diff: core/sql/comexe/ComTdbExeUtil.cpp ---
@@ -2231,28 +2233,32 @@ ComTdbExeUtilLobExtract::ComTdbExeUtilLobExtract
handle_(handle),
handleLen_(handleLen),
toType_((short)toType),
+ bufAddr_(bufAddr),
+ extractSizeIOAddr_(extractSizeAddr),
lobStorageType_(lobStorageType),
stringParam1_(stringParam1),
stringParam2_(stringParam2),
stringParam3_(stringParam3),
lobHdfsServer_(lobHdfsServer),
lobHdfsPort_(lobHdfsPort),
- rowSize_(0),
- bufSize_(0),
+ totalBufSize_(0),
flags_(0)
{
setNodeType(ComTdb::ex_LOB_EXTRACT);
if ((toType_ == ExtractToType::TO_BUFFER_) || (toType_ == ExtractToType::TO_STRING_))
{
- // intparam1 contains the rowsize passed in via syntax
+ /*// intparam1 contains the rowsize passed in via syntax
// intparam2 constains the total buf size user has allocated
rowSize_ = intParam1;
- bufSize_ = intParam2;
+ bufSize_ = intParam2; */
--- End diff --
Deleted
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48911574
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**********************************************************************
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**********************************************************************/
+/* -*-C++-*-
+****************************************************************************
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description: Test driver useing exe util cli interface
+*
+*
+*
+*
+****************************************************************************
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle,
+ Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ char * query = new char[4096];
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ //Use lob handle to retrieve the lob length.
+ char lobLengthResult[200];
+ str_cpy_all(lobLengthResult," ",200);
+ Int32 lobLengthResultLen = 0;
+ str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld ",lobHandle, &lengthOfLob);
+ retcode = cliInterface.executeImmediate(query,lobLengthResult,&lobLengthResultLen,FALSE);
+
+ delete query;
+ return retcode;
+
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ char * query = new char[4096];
+ Int32 lobHandleLen = 0;
+ str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+
+ retcode = cliInterface.executeImmediate(query,lobHandle,&lobHandleLen,FALSE);
+
+ if (retcode)
+ return retcode;
--- End diff --
Leaks the query memory
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by sandhyasun <gi...@git.apache.org>.
Github user sandhyasun commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/248#discussion_r48979445
--- Diff: core/sql/clitest/blobtest.cpp ---
@@ -0,0 +1,255 @@
+/**********************************************************************
+// @@@ START COPYRIGHT @@@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// @@@ END COPYRIGHT @@@
+**********************************************************************/
+/* -*-C++-*-
+****************************************************************************
+*
+* File: Helper functions for use by bin/clitest.cpp
+* Description: Test driver useing exe util cli interface
+*
+*
+*
+*
+****************************************************************************
+*/
+#include "blobtest.h"
+
+Int32 extractLengthOfLobColumn(CliGlobals *cliglob, char *lobHandle,
+ Int64 &lengthOfLob,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ char * query = new char[4096];
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ //Use lob handle to retrieve the lob length.
+ char lobLengthResult[200];
+ str_cpy_all(lobLengthResult," ",200);
+ Int32 lobLengthResultLen = 0;
+ str_sprintf(query,"extract loblength (lob '%s') LOCATION %Ld ",lobHandle, &lengthOfLob);
+ retcode = cliInterface.executeImmediate(query,lobLengthResult,&lobLengthResultLen,FALSE);
+
+ delete query;
+ return retcode;
+
+}
+
+Int32 extractLobHandle(CliGlobals *cliglob, char *& lobHandle,
+ char *lobColumnName, char *tableName)
+{
+ Int32 retcode = 0;
+ ExeCliInterface cliInterface((cliglob->currContext())->exHeap(), (Int32)SQLCHARSETCODE_UTF8, cliglob->currContext(),NULL);
+ char * query = new char[4096];
+ Int32 lobHandleLen = 0;
+ str_sprintf(query,"select %s from %s",lobColumnName,tableName);
+
+ retcode = cliInterface.executeImmediate(query,lobHandle,&lobHandleLen,FALSE);
+
+ if (retcode)
+ return retcode;
--- End diff --
For the syntax in the previous comment(could not add Line Note there) , it was because the comments use markup and all test within <> got chopped off. The syntax should look better now, I edited it.
Fixed memory leak .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Support for extracting lob to bu...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-trafodion/pull/248
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---