You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Kristian Waagan (JIRA)" <ji...@apache.org> on 2010/01/26 08:56:34 UTC
[jira] Created: (DERBY-4525) Document the in-memory storage back
end
Document the in-memory storage back end
---------------------------------------
Key: DERBY-4525
URL: https://issues.apache.org/jira/browse/DERBY-4525
Project: Derby
Issue Type: Task
Components: Documentation
Affects Versions: 10.6.0.0
Reporter: Kristian Waagan
Fix For: 10.6.0.0
The in-memory back end isn't considered experimental anymore, we have to
write user documentation for the feature(s).
I'm not sure how it should be structured, and where the content should be added.
Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
- documenting the new protocol name ('memory')
- documenting the new 'drop' JDBC connection URL attribute
- describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
- "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
- tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
- known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843514#action_12843514 ]
Knut Anders Hatlen commented on DERBY-4525:
-------------------------------------------
It sounds like a bug to me if you're able to drop a file-system database on the server. (I didn't get that message myself when I tried the same commands on trunk. Instead, it looked like the drop attribute was ignored, and a new connection was opened. Still sounds like a bug, though.)
The difference in the error messages comes from the way we process error messages on the client. In the normal case, the server sends the message id, the message translated to the server's locale and the arguments used to generate the message. The client passes this information and information about its own locale back to the server by invoking a stored procedure. The server then translates the message to the client's locale so that it can be displayed correctly. However, when CONNECT fails, there is no connection on which the stored procedure can be invoked in order to translate the message, so the client just dumps whatever information it got from the server in the first run.
So, yes, the difference is expected, but probably not desirable. I'm not sure if we have a JIRA tracking this problem.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12844696#action_12844696 ]
Kim Haase commented on DERBY-4525:
----------------------------------
Well, the wrong-format error message may not be that serious an issue. But it does seem to be the case that you can drop an on-disk database, at least if you create it in a directory called "memory".
I (accidentally, because of a punctuation error) create a db called myDB in the directory memory:
ij> connect 'jdbc:derby://localhost:1527/memory/myDB;create=true';
ij> create table t(c int);
0 rows inserted/updated/deleted
The database is visible in the file system:
ls ~/DERBYTUTOR/*
/home/chaase/DERBYTUTOR/derby.log
/home/chaase/DERBYTUTOR/firstdb:
log/ seg0/ service.properties
/home/chaase/DERBYTUTOR/memory:
myDB/
/home/chaase/DERBYTUTOR/seconddb:
log/ seg0/ service.properties
The database is actually there:
ls ~/DERBYTUTOR/memory/myDB
db.lck log/ service.properties
dbex.lck seg0/ tmp/
I am then able to drop this database even though it's a file system db:
ij> connect 'jdbc:derby://localhost:1527/memory/myDB;drop=true';
ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory/myDB' dropped.
The database is gone:
ls ~/DERBYTUTOR/*
/home/chaase/DERBYTUTOR/derby.log
/home/chaase/DERBYTUTOR/firstdb:
log/ seg0/ service.properties
/home/chaase/DERBYTUTOR/memory:
/home/chaase/DERBYTUTOR/seconddb:
log/ seg0/ service.properties
I then try to drop another filesystem db, one that doesn't have "memory" in the name. This fails, as it should:
ij> connect 'jdbc:derby://localhost:1527/seconddb;drop=true';
ERROR XBM0I: DERBY SQL error: SQLCODE: -1, SQLSTATE: XBM0I, SQLERRMC: Directory seconddb cannot be removed.
I then use the correct syntax to create the in-memory db. It is created, but in a new connection.
ij> connect 'jdbc:derby://localhost:1527/memory:myDB;create=true';
ij(CONNECTION1)> show connections;
CONNECTION0 - jdbc:derby://localhost:1527/memory/myDB;create=true
CONNECTION1* - jdbc:derby://localhost:1527/memory:myDB;create=true
* = current connection
I find that if I start over and create the filesystem db, I can shut it down and it stays there:
ij> connect 'jdbc:derby://localhost:1527/memory/myDB;create=true';
ij> connect 'jdbc:derby://localhost:1527/memory/myDB;shutdown=true';
ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory/myDB' shutdown.
ls ~/DERBYTUTOR/memory
myDB/
This is as it should be. I shouldn't be able to drop it, though.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843174#action_12843174 ]
Knut Anders Hatlen commented on DERBY-4525:
-------------------------------------------
Hi Kim,
DERBY-4428 contains more information about the new 'drop' attribute.
+1 to remove the sections that only apply to JVMs that we no longer support, but that's probably better to address in a separate issue. (We may want to keep the first two paragraphs in some form to make sure that it's documented that you cannot boot two instances of the same database simultaneously.)
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843248#action_12843248 ]
Kim Haase commented on DERBY-4525:
----------------------------------
Actually, that syntax was wrong -- I accidentally put a slash instead of a colon, which created a file-system database named myDB in the directory memory. I shouldn't have been able to drop it, should I? I thought drop only worked on in-memory databases.
The same error results if I do it right.
ij> connect 'jdbc:derby://localhost:1527/memory:myDB;create=true';
ij> connect 'jdbc:derby://localhost:1527/memory:myDB;drop=true';
ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory:myDB' dropped.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-4525) Document the in-memory storage back end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kim Haase closed DERBY-4525.
----------------------------
Closing, since changes now appear in Latest Alpha Manuals.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
> Attachments: DERBY-4525-2.diff, DERBY-4525-2.stat, DERBY-4525-2.zip, DERBY-4525-3.diff, DERBY-4525-3.zip, DERBY-4525.diff, DERBY-4525.stat, DERBY-4525.zip
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kim Haase updated DERBY-4525:
-----------------------------
Attachment: DERBY-4525.zip
DERBY-4525.stat
DERBY-4525.diff
Here's a first pass at a patch. Attaching DERBY-4525.diff, DERBY-4525.stat, and DERBY-4525.zip, with changes to 24 files (including the map files) and 5 of the 6 manuals. Most changes are relatively small. Here's a summary.
Admin Guide:
cadminappsclient.dita: Accessing the Network Server by using the network client driver
Add memory to syntax statement and explain (w. xref to cdevdvlpinmemdb.dita).
Dev Guide:
cdevdvlp17453.dita: Derby JDBC database connection URL
Add memory to subsubprotocol list
cdevdvlp27610.dita: Derby system
Mention that use of in-memory db means that no databases exist in the directory.
cdevdvlp96597.dita: One Derby instance for each Java Virtual Machine
Revise.
Mention that if you specify the "same" in-memory db in two different instances of Derby it is not a problem because you actually get two different dbs in two different JVMs.
cdevdvlp19297.dita: Recommended practices
Fix next-to-last bullet to remove the recommendation against doing things that are actually impossible to do.
cdevdvlp40724.dita: The database directory
Mention that the directory info doesn't apply to in-memory dbs. I'm guessing the size limits still apply -- let me know if not.
cdevdvlp42173.dita: Creating, dropping, and backing up databases
You can drop an in-memory db.
cdevdvlp18166.dita: Storage and recovery
Doesn't apply to in-memory dbs, does it?
cdevdvlp846369.dita: Connecting to databases within the system
Move the verb "is" to make it clear that this info is about file-system dbs only.
cdevdvlp40350.dita: Conventions for specifying the database path
Add memory to list of non-filesystem access methods.
cdevdvlp19700.dita: Special database access
Add note on in-memory and xref.
rdevdvlp22102.dita: Database connection examples
Add example of in-memory.
cdevdvlpinmemdb.dita: Using in-memory databases
New topic. Does placement seem okay?
Ref Man:
rrefattrib24612.dita: Setting attributes for the database connection URL
Add mention of subsubprotocol
rrefattrib26867.dita: create=true attribute
Add examples of creating in-memory db.
rrefattribdrop.dita: drop=true attribute
New topic.
rrefattrib16471.dita: shutdown=true attribute
Add examples of shutting down in-memory db.
Tools Guide:
rtoolsijcomref27997.dita: Protocol command
Add example of connecting to in-memory db.
rtoolsijcomref22318.dita: Connect command
Add example of connecting to in-memory db. Also add example of using PROTOCOL in command
Tuning Guide:
ctunperfinmemdb.dita: Configure Derby to use an in-memory database
New topic.
ctunperf25864.dita: The tips
Add mention of tuning for in-memory db. Also corrected title of table-function topic, here and in map file.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
> Attachments: DERBY-4525.diff, DERBY-4525.stat, DERBY-4525.zip
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845295#action_12845295 ]
Knut Anders Hatlen commented on DERBY-4525:
-------------------------------------------
The problem with databaseName looks like a bug.
As to the replication properties, I think you are right that we haven't done anything actively to support replication of in-memory databases. That said, I was able to set up replication between two memory databases and issue a fail-over between them, and all the operations appeared to work as expected, so I'm not sure if the docs need to say anything special about the combination memory/replication.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kim Haase resolved DERBY-4525.
------------------------------
Issue & fix info: (was: [Patch Available])
Resolution: Fixed
After getting verbal okay (thanks, Kristian), committed patch DERBY-4525-3.diff to documentation trunk at revision 936885.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
> Attachments: DERBY-4525-2.diff, DERBY-4525-2.stat, DERBY-4525-2.zip, DERBY-4525-3.diff, DERBY-4525-3.zip, DERBY-4525.diff, DERBY-4525.stat, DERBY-4525.zip
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kim Haase updated DERBY-4525:
-----------------------------
Issue & fix info: [Patch Available]
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
> Attachments: DERBY-4525.diff, DERBY-4525.stat, DERBY-4525.zip
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858892#action_12858892 ]
Kristian Waagan commented on DERBY-4525:
----------------------------------------
Hi Kim,
Thanks for writing these docs. I think they look good, and they can be committed
as they are if that is required to get them included in 10.6.
I have made some comments and suggestions below. Please use you own
judgment regarding whether they are worth implementing or not.
* src/devguide/cdevdvlp19700.dita
Should we add a few words to clearify that you can use the in-memory back end in
other cases than testing and devlopment, for instance when you are processing
transient/reproducible data?
If you have the required memory avaialble, the benefits are (or may be) faster
processing (no disk IO) and simplicity when it comes to discarding the data. If
changed, revisit cdevdvlpinmemdb.html.
* src/devguide/cdevdvlp40724.dita
Should we mention that for some of the limits the maximum value is determined by
the available main memory instead of disk space / file system, or is this too
obvious to mention?
* src/devguide/cdevdvlpinmemdb.dita
Removing an in-memory database: Does it make sense to swap the order of the
current content, i.e. to describe how you would actively remove an in-memory db
in a running JVM first?
To me, the fact that in-memory databases are lost when the JVM exits is more
like a side-effect. We do need to mention this, but maybe more like a word of
caution to make it clear to users that in-memory databases are not persisted.
Mention that if authentication and SQL authorization is enabled, only the
database owner can drop a database. I'm not sure if we have done this for other
actions, like shutting down the database.
* src/devguide/rdevdvlp22102.dita
Add a link to a page describing, or just a note, that using the in-memory back
end with the client driver requires a "special" URL (include the subsubprotocol
as part of the name)?
* src/ref/rrefattrib16471.dita, src/ref/rrefattrib26867.dita
Use the same database name for embedded and client/server for consistency in the
examples? (currently 'myInMemDB' and 'MyInMemDB')
* src/ref/rrefattrib17246.dita
In the example:
"+<codeblock><b>jdbc:derby:toursDB;databaseName=flightsDB</b></codeblock>" - specifies two database names
* src/ref/rrefattribdrop.dita
"This attribute, like shutdown=true, cannot be combined with other attributes." - with the exception of user credentials?
Mention that using the drop attribute with other back ends results in an
exception with SQLState XBM0I to be thrown?
* src/tools/rtoolsijcomref22318.dita and src/tools/rtoolsijcomref27997.dita
Would it make sense to show that you can set the memory subsubprotocol as part
of the protocol to create in-memory databases by default in ij?
I.e:
ij> protocol 'jdbc:derby:memory:';
ij> connect 'testDB;create=true';
* src/tuning/ctunperfinmemdb.dita
I would maybe change the wording a bit for the page cache size. There is nothing
wrong with running with a smaller than default page cache, but the user may see
poor performance. You don't get the penalty for going to the disk to fetch
pages, but if the pressure on the page cache gets too high you will get a lot of
contention and increased CPU usage. To me, the current wording suggests that I
should increase the page cache size when using the in-memory db. In many cases
this is not necessary.
Can we say that we don't recommend to run with a smaller than default page
cache? (keep the sentence about data passing through the cache)
Finally, should we mention that users can use the backup routines to persist an
in-memory database? This backup can then be booted as an in-memory database
again at a later time, or simply be used as a normal disk based db. Or does
suggesting this use case make some people uncomfortable?
Thanks,
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
> Attachments: DERBY-4525-2.diff, DERBY-4525-2.stat, DERBY-4525-2.zip, DERBY-4525.diff, DERBY-4525.stat, DERBY-4525.zip
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843070#action_12843070 ]
Kim Haase commented on DERBY-4525:
----------------------------------
I have a couple of questions -- first, about the list of topics above: What is "the new 'drop' JDBC connection URL attribute"? I cannot find it in either DERBY-646 or http://wiki.apache.org/db-derby/InMemoryBackEndPrimer.
Also, I have been going through the Developer's Guide (http://db.apache.org/derby/docs/dev/devguide/) checking for topics that need to have information about the in-memory back end added. I think something should be said in "Derby system" and some of its subtopics. "One Derby instance for each Java Virtual Machine" (http://db.apache.org/derby/docs/dev/devguide/cdevdvlp96597.html) has a link to "Double-booting system behavior" (http://db.apache.org/derby/docs/dev/devguide/cdevdvlp20458.html). This second topic seems to be entirely obsolete, since it describes problems that can happen with a JVM earlier than 1.4.2, and Derby no longer supports such JVMs. Only the very last paragraph, about using the Network Server if you want more than one Derby instance to be able to access a database, is still current. Would it make sense to remove this topic from the manual and move this last paragraph to the topic "One Derby instance for each Java Virtual Machine"?
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kim Haase updated DERBY-4525:
-----------------------------
Attachment: DERBY-4525-2.diff
DERBY-4525-2.stat
DERBY-4525-2.zip
Attaching a revised patch, DERBY-4525-2.diff, DERBY-4525-2.stat, and DERBY-4525-2.zip. This includes changes to one additional file, rrefattrib17246.dita, on the databaseName attribute, now that we know how to use it to specify an in-memory database (see DERBY-4581). I also added an example of the use of the databaseName attribute to the create attribute topic, and corrected the dev guide map file to reflect the title change of one of the dev guide topics.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
> Attachments: DERBY-4525-2.diff, DERBY-4525-2.stat, DERBY-4525-2.zip, DERBY-4525.diff, DERBY-4525.stat, DERBY-4525.zip
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kim Haase reassigned DERBY-4525:
--------------------------------
Assignee: Kim Haase
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12844754#action_12844754 ]
Kim Haase commented on DERBY-4525:
----------------------------------
I wonder if we need to say anything about what connection attributes work with the in-memory db and which don't. I think in most cases it would be intuitive -- attributes having to do with replication wouldn't, would they? I would think most others would.
One that I would expect to work doesn't though -- databaseName. When I'm in ij, I can create an in-memory db, disconnect, and then connect again, and the db is still there. I can connect to it the usual way, but not using the databaseName attribute -- I tried a couple of different ways:
ij> connect 'jdbc:derby:memory:myDB;create=true';
ij> create table t(c int);
0 rows inserted/updated/deleted
ij> disconnect;
ij> connect 'jdbc:derby:memory:myDB';
ij> insert into t values(1), (2);
2 rows inserted/updated/deleted
ij> select * from t;
C
-----------
1
2
2 rows selected
ij> disconnect;
ij> connect 'jdbc:derby:memory;databaseName=myDB';
ERROR XJ004: Database 'memory' not found.
ij> connect 'jdbc:derby:memory:;databaseName=myDB';
ERROR XJ004: Database 'memory:' not found.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843183#action_12843183 ]
Kim Haase commented on DERBY-4525:
----------------------------------
Thanks, Knut! I've linked to DERBY-4428 to have that info handy.
It is true that the issues in the two topics are different -- thanks for pointing that out. "One Derby instance for each Java Virtual Machine" is not the same thing as "One Derby instance for each database". So they should both stay. Maybe I'll file and fix the separate issue about removing the obsolete info from that topic first, before I deal with the in-memory DB doc.
One question about DERBY-4428 -- I notice when I try out the drop attribute in ij I get the following error:
ij> CONNECT 'jdbc:derby:memory:dummy;drop=true';
ERROR 08006: Database 'memory:dummy' dropped.
This is the expected not-really-error when you shut down a database, so I guess it is getting yet one more overload -- currently there are 8 messages associated with 08006 and now I guess there will be one more. It seems to appear both when you shut the db down and when you drop it. Is that correct?
It doesn't seem to be necessary to shut the db down before you drop it -- the drop seems to accomplish both.
It appears that you can only use the drop attribute to drop an in-memory database, not a file-system one. If I try the latter I get
ij> connect 'jdbc:derby:firstdb;drop=true';
ERROR XBM0I: Directory firstdb cannot be removed.
That seems like a useful safeguard.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845328#action_12845328 ]
Kim Haase commented on DERBY-4525:
----------------------------------
Thanks for the advice, Knut. I will file an issue on the databaseName problem and, in general, not say anything about the in-memory database and connection attributes (except to add examples for the most common ones like create and shutdown).
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843222#action_12843222 ]
Kim Haase commented on DERBY-4525:
----------------------------------
I find that if I use the Network Server and I drop an in-memory database, the error is a bit different from the embedded one:
ij> connect 'jdbc:derby://localhost:1527/memory/myDB;create=true';
ij> create table t(c int);
0 rows inserted/updated/deleted
ij> connect 'jdbc:derby://localhost:1527/memory/myDB;drop=true';
ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory/myDB' dropped.
Is this expected?
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4525) Document the in-memory storage back
end
Posted by "Kim Haase (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kim Haase updated DERBY-4525:
-----------------------------
Attachment: DERBY-4525-3.diff
DERBY-4525-3.zip
Thanks so much, Kristian, for the careful review. I'm attaching a final(?) patch, DERBY-4525-3.diff and DERBY-4525-3.zip. I hope I have incorporated all your edits. A couple of notes:
* src/ref/rrefattrib17246.dita
In the example:
"+<codeblock><b>jdbc:derby:toursDB;databaseName=flightsDB</b></codeblock>" -
specifies two database names
Actually, this is intentional -- the description above is of what happens if you specify two names this way. But perhaps it is an obscure enough case that it should come after the in-memory db info -- so I swapped the order.
I hope the wording in the Tuning Guide topic is okay -- I didn't add all the supporting detail, just changed the emphasis. Let me know if it still needs work.
I added a small section on persisting databases to the Dev Guide topic. If anyone objects, we can change it.
I changed the wording of the note in "Derby system" (src/devguide/cdevdvlp27610.dita) slightly.
> Document the in-memory storage back end
> ---------------------------------------
>
> Key: DERBY-4525
> URL: https://issues.apache.org/jira/browse/DERBY-4525
> Project: Derby
> Issue Type: Task
> Components: Documentation
> Affects Versions: 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kim Haase
> Fix For: 10.6.0.0
>
> Attachments: DERBY-4525-2.diff, DERBY-4525-2.stat, DERBY-4525-2.zip, DERBY-4525-3.diff, DERBY-4525-3.zip, DERBY-4525.diff, DERBY-4525.stat, DERBY-4525.zip
>
>
> The in-memory back end isn't considered experimental anymore, we have to
> write user documentation for the feature(s).
> I'm not sure how it should be structured, and where the content should be added.
> Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):
> - documenting the new protocol name ('memory')
> - documenting the new 'drop' JDBC connection URL attribute
> - describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
> - "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
> - tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
> - known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)
> Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.