You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Bryan Pendleton (JIRA)" <de...@db.apache.org> on 2006/08/09 00:07:14 UTC
[jira] Updated: (DERBY-688) Enhancements to XML functionality to
move toward XPath/XQuery support...
[ http://issues.apache.org/jira/browse/DERBY-688?page=all ]
Bryan Pendleton updated DERBY-688:
----------------------------------
Derby Info: (was: [Patch Available])
Committed d688_phase3_v1_code.patch and d688_phase3_v1_tests.patch
to subversion as revision 429847. Committed the two patches together per
Army's recommendation to commit these patches as a unit to avoid test diffs.
Clearing the patch available flag because all the pending patches have now been committed.
Thanks for all the hard work on this, Army!
> Enhancements to XML functionality to move toward XPath/XQuery support...
> ------------------------------------------------------------------------
>
> Key: DERBY-688
> URL: http://issues.apache.org/jira/browse/DERBY-688
> Project: Derby
> Issue Type: Improvement
> Components: SQL, JDBC
> Reporter: A B
> Assigned To: A B
> Priority: Minor
> Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, d688_phase3_v1_tests.patch, derbyXMLSpec.html
>
>
> As of DERBY-334, Derby has some very basic support for XML that consists of an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). I would like to enhance this existing functionality and, by doing so, help to move Derby incrementally toward a more usable and more complete XPath/XQuery solution (with emphasis on "incrementally").
> I have attached to this issue a document describing the particular changes that I am looking to make. At a high level, they consist of:
> 1) Making it easier to use the XML operators and datatype from within JDBC (ex. by implicit parsing/serialization of XML values).
> 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results of an XPath expression (instead of just determining whether or not the expression evaluates to an empty sequence, which is what XMLEXISTS does).
> 3) Making changes to the existing operators to line them up with the SQL/XML 2005 specification, and also to take steps toward my eventual hope of having support for XQuery (as opposed to just XPath) in Derby.
> If anyone has time and interest enough to look at the document and provide feedback, that'd be great...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Updated: (DERBY-688) Enhancements to XML functionality
to move toward XPath/XQuery support...
Posted by Army <qo...@gmail.com>.
David Van Couvering wrote:
> and thanks for the careful review and effort by Brian and Yip!
Agreed!
> I am telling people "we have XML features" and I know it's
> something to do with XQuery and XPath, but I couldn't say what.
>
> Did these patches include documentation? Or is that forthcoming?
Documentation is still forthcoming, as are some additional patches to finalize
the work. There're still a couple of tasks to complete--such as enabling tests
to run as part of derbyall--but the bulk of it is now in the codeline.
Documentation is one of the remaining tasks.
Army
Re: XML in Derby - WAS : [jira] Updated: (DERBY-688) Enhancements
to XML functionality to move toward XPath/XQuery support...
Posted by Daniel John Debrunner <dj...@apache.org>.
Kristian Waagan wrote:
> Daniel John Debrunner wrote:
>
>> David Van Couvering wrote:
>>
>>> Wow, great work Army, and thanks for the careful review and effort by
>>> Brian and Yip! It's great to have this in. If I could only grok
>>> exactly what the feature is and how I might use it :) I am telling
>>> people "we have XML features" and I know it's something to do with
>>> XQuery and XPath, but I couldn't say what.
>>
>>
>> I was messing with the XML support last night for buddy testing and it's
>> very cool. Three new Derby features combined together to make the
>> application development easier:
>>
>> 1) XML support
>> 2) CALL procedure in trigger
>> 3) Lengthless overrides for PreparedStatement.setCharacterStream
>>
>> OK - I didn't test 3) cos I would have to set up Xalan for Mustang and I
>> didn't want to spend time on how to figure that out, but it would have
>> been useful.
>
>
> Hi Dan,
>
> This sounds really cool :)
> Any change you can share your trigger procedure?
> Then maybe someone else can take on the work to test the new lengthless
> overrides added by JDBC 4.0. I don't think they have received much
> testing yet. They are now in the 10.2 branch, but I believe they are not
> (fully) included in the latest beta (10.2.1.1). I'll add an entry to the
> buddy testing page when the time is right.
Here's the Java method:
public static void get_url_content(String id, String path)
throws SQLException, IOException
{
Connection conn =
DriverManager.getConnection("jdbc:default:connection");
PreparedStatement ps = conn.prepareStatement(
"UPDATE WDD.WEB_DOCS SET WD_CONTENT = " +
"XMLPARSE (DOCUMENT CAST (? AS CLOB) PRESERVE WHITESPACE)" +
" , WD_ACCESSTIME = CURRENT TIMESTAMP " +
"WHERE WD_ID = ?");
ps.setString(2, id);
URL url = new URL(path);
URLConnection urlConn = url.openConnection();
urlConn.connect();
int length = urlConn.getContentLength();
String enc = urlConn.getContentEncoding();
if (enc == null)
enc = "UTF-8";
InputStream in = urlConn.getInputStream();
InputStreamReader isr = new InputStreamReader(in, enc);
// HACK - Assume number of characters will be
// the same as the number of bytes.
ps.setCharacterStream(1, isr, length);
ps.execute();
in.close();
ps.close();
conn.close();
}
and the SQL
DROP TABLE WDD.WEB_DOCS;
CREATE TABLE WDD.WEB_DOCS (
WD_ID VARCHAR(128) PRIMARY KEY,
WD_URL VARCHAR(1000),
WD_CONTENT XML,
WD_ACCESSTIME TIMESTAMP
);
DROP PROCEDURE WDD.GET_URL_CONTENT;
CREATE PROCEDURE WDD.GET_URL_CONTENT(ID VARCHAR(128), URL VARCHAR(1000))
LANGUAGE JAVA PARAMETER STYLE JAVA
EXTERNAL NAME 'wdd.get_url_content';
CREATE TRIGGER WDD.WD_I AFTER INSERT
ON WDD.WEB_DOCS
REFERENCING NEW AS NEW
FOR EACH ROW MODE DB2SQL
CALL WDD.GET_URL_CONTENT(NEW.WD_ID, NEW.WD_URL);
Dan.
Re: XML in Derby - WAS : [jira] Updated: (DERBY-688) Enhancements to
XML functionality to move toward XPath/XQuery support...
Posted by Kristian Waagan <Kr...@Sun.COM>.
Daniel John Debrunner wrote:
> David Van Couvering wrote:
>
>> Wow, great work Army, and thanks for the careful review and effort by
>> Brian and Yip! It's great to have this in. If I could only grok
>> exactly what the feature is and how I might use it :) I am telling
>> people "we have XML features" and I know it's something to do with
>> XQuery and XPath, but I couldn't say what.
>
> I was messing with the XML support last night for buddy testing and it's
> very cool. Three new Derby features combined together to make the
> application development easier:
>
> 1) XML support
> 2) CALL procedure in trigger
> 3) Lengthless overrides for PreparedStatement.setCharacterStream
>
> OK - I didn't test 3) cos I would have to set up Xalan for Mustang and I
> didn't want to spend time on how to figure that out, but it would have
> been useful.
Hi Dan,
This sounds really cool :)
Any change you can share your trigger procedure?
Then maybe someone else can take on the work to test the new lengthless
overrides added by JDBC 4.0. I don't think they have received much
testing yet. They are now in the 10.2 branch, but I believe they are not
(fully) included in the latest beta (10.2.1.1). I'll add an entry to the
buddy testing page when the time is right.
Regards,
--
Kristian
>
> I setup a table with a column that contained a URL as a VARCHAR and an
> XML column, a procedure in an INSERT trigger then fetched the data from
> the URL and updated the XML column using XMLPARSE, streaming directly
> from the remote site using setCharacterStream.
>
> I used this to download DERBY Jira issues, each row holds an XML
> document that corresponds to a single Jira issue. E.g. from
>
> http://issues.apache.org/jira/browse/DERBY-434?decorator=none&view=rss
>
> Then I can execute queries against the issues locally, using XPath (and
> SQL).
>
> -- Sequence of all comments made by Sunitha against bugs reported by me
> -- wd_id is the DERBY-XXX identifier
> -- wd_accesstime is the time the data was downloaded from the web.
> -- wd_content is the XML column
>
> select wd_id, wd_accesstime,
> XMLSERIALIZE(
> XMLQUERY('//item/comments/comment[@author="skambha"]' PASSING BY REF
> wd_content EMPTY ON EMPTY)
> AS VARCHAR(30000))
> from wdd.web_docs where
> XMLEXISTS('//reporter[text() = "Daniel John Debrunner"]' PASSING BY REF
> wd_content);
>
> -- Jira status of all bugs entered by me
>
> select wd_id,
> XMLSERIALIZE(
> XMLQUERY('//item/status/text()' PASSING BY REF wd_content EMPTY ON EMPTY)
> AS VARCHAR(20)),
> wd_accesstime
> from wdd.web_docs where
> XMLEXISTS('//reporter[text() = "Daniel John Debrunner"]' PASSING BY REF
> wd_content)
> order by 2,1;
>
> I'll think I wil expand this to use it in my talk at ApacheCon US.
> Dan.
>
>
>
>
>
>
>
>
XML in Derby - WAS : [jira] Updated: (DERBY-688) Enhancements to
XML functionality to move toward XPath/XQuery support...
Posted by Daniel John Debrunner <dj...@apache.org>.
David Van Couvering wrote:
> Wow, great work Army, and thanks for the careful review and effort by
> Brian and Yip! It's great to have this in. If I could only grok
> exactly what the feature is and how I might use it :) I am telling
> people "we have XML features" and I know it's something to do with
> XQuery and XPath, but I couldn't say what.
I was messing with the XML support last night for buddy testing and it's
very cool. Three new Derby features combined together to make the
application development easier:
1) XML support
2) CALL procedure in trigger
3) Lengthless overrides for PreparedStatement.setCharacterStream
OK - I didn't test 3) cos I would have to set up Xalan for Mustang and I
didn't want to spend time on how to figure that out, but it would have
been useful.
I setup a table with a column that contained a URL as a VARCHAR and an
XML column, a procedure in an INSERT trigger then fetched the data from
the URL and updated the XML column using XMLPARSE, streaming directly
from the remote site using setCharacterStream.
I used this to download DERBY Jira issues, each row holds an XML
document that corresponds to a single Jira issue. E.g. from
http://issues.apache.org/jira/browse/DERBY-434?decorator=none&view=rss
Then I can execute queries against the issues locally, using XPath (and
SQL).
-- Sequence of all comments made by Sunitha against bugs reported by me
-- wd_id is the DERBY-XXX identifier
-- wd_accesstime is the time the data was downloaded from the web.
-- wd_content is the XML column
select wd_id, wd_accesstime,
XMLSERIALIZE(
XMLQUERY('//item/comments/comment[@author="skambha"]' PASSING BY REF
wd_content EMPTY ON EMPTY)
AS VARCHAR(30000))
from wdd.web_docs where
XMLEXISTS('//reporter[text() = "Daniel John Debrunner"]' PASSING BY REF
wd_content);
-- Jira status of all bugs entered by me
select wd_id,
XMLSERIALIZE(
XMLQUERY('//item/status/text()' PASSING BY REF wd_content EMPTY ON EMPTY)
AS VARCHAR(20)),
wd_accesstime
from wdd.web_docs where
XMLEXISTS('//reporter[text() = "Daniel John Debrunner"]' PASSING BY REF
wd_content)
order by 2,1;
I'll think I wil expand this to use it in my talk at ApacheCon US.
Dan.
Re: [jira] Updated: (DERBY-688) Enhancements to XML functionality to
move toward XPath/XQuery support...
Posted by David Van Couvering <Da...@Sun.COM>.
Wow, great work Army, and thanks for the careful review and effort by
Brian and Yip! It's great to have this in. If I could only grok
exactly what the feature is and how I might use it :) I am telling
people "we have XML features" and I know it's something to do with
XQuery and XPath, but I couldn't say what.
Did these patches include documentation? Or is that forthcoming?
David
Bryan Pendleton (JIRA) wrote:
> [ http://issues.apache.org/jira/browse/DERBY-688?page=all ]
>
> Bryan Pendleton updated DERBY-688:
> ----------------------------------
>
> Derby Info: (was: [Patch Available])
>
> Committed d688_phase3_v1_code.patch and d688_phase3_v1_tests.patch
> to subversion as revision 429847. Committed the two patches together per
> Army's recommendation to commit these patches as a unit to avoid test diffs.
>
> Clearing the patch available flag because all the pending patches have now been committed.
>
> Thanks for all the hard work on this, Army!
>
>
>> Enhancements to XML functionality to move toward XPath/XQuery support...
>> ------------------------------------------------------------------------
>>
>> Key: DERBY-688
>> URL: http://issues.apache.org/jira/browse/DERBY-688
>> Project: Derby
>> Issue Type: Improvement
>> Components: SQL, JDBC
>> Reporter: A B
>> Assigned To: A B
>> Priority: Minor
>> Attachments: d688_phase1_v1.patch, d688_phase1_v1.stat, d688_phase1_v2.patch, d688_phase1_v3.patch, d688_phase2_v1_code.patch, d688_phase2_v1_tests.patch, d688_phase2_v2_tests.patch, d688_phase2_v3_tests.patch, d688_phase3_v1_code.patch, d688_phase3_v1_tests.patch, derbyXMLSpec.html
>>
>>
>> As of DERBY-334, Derby has some very basic support for XML that consists of an XML datatype and three operators (XMLPARSE, XMLSERIALIZE, and XMLEXISTS). I would like to enhance this existing functionality and, by doing so, help to move Derby incrementally toward a more usable and more complete XPath/XQuery solution (with emphasis on "incrementally").
>> I have attached to this issue a document describing the particular changes that I am looking to make. At a high level, they consist of:
>> 1) Making it easier to use the XML operators and datatype from within JDBC (ex. by implicit parsing/serialization of XML values).
>> 2) Adding a new operator, XMLQUERY, to allow a user to retrieve the results of an XPath expression (instead of just determining whether or not the expression evaluates to an empty sequence, which is what XMLEXISTS does).
>> 3) Making changes to the existing operators to line them up with the SQL/XML 2005 specification, and also to take steps toward my eventual hope of having support for XQuery (as opposed to just XPath) in Derby.
>> If anyone has time and interest enough to look at the document and provide feedback, that'd be great...
>