You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@labs.apache.org by ka...@apache.org on 2009/03/14 12:36:59 UTC
svn commit: r753638 - in /labs/bananadb/trunk: ./
src/main/java/org/apache/labs/bananadb/hashtable/
src/main/java/org/apache/labs/bananadb/index/ src/test/java/
src/test/java/org/apache/labs/bananadb/demo/
src/test/java/org/apache/labs/bananadb/hashtable/
Author: kalle
Date: Sat Mar 14 11:36:58 2009
New Revision: 753638
URL: http://svn.apache.org/viewvc?rev=753638&view=rev
Log:
BananaDB
Rehashing, growing hashtable capacity.
Hashtable and Key postings file now both got a header of 1024 bytes
that contains start and end offset. On rehash a write lock is set
and the hashtable and key postings files are appended with null data
on which the rehashed postings are written.
When rehash finish, the start- and end offset in the file headers are updated
and thus any JVM running using the DB will start reading from the rehashed data.
In the future one should implement
1. A cleanup() method that removes old posting parts of the files.
2. Automatic rehasing when the next available offset hits a threadshold.
CAVEAT!
Cursors iterating values will throw an exception in case there has been a rehash.
Added:
labs/bananadb/trunk/CHANGES.txt
labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/index/README.txt
labs/bananadb/trunk/src/test/java/log4j.properties
labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/HashtableTest.java
labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestMultiThreaded.java
- copied, changed from r753612, labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java
labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestRehash.java
- copied, changed from r753612, labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java
labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestValuePartitions.java
- copied, changed from r753612, labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java
Removed:
labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/demo/
Modified:
labs/bananadb/trunk/FILES.txt
labs/bananadb/trunk/README.txt
labs/bananadb/trunk/TODO.txt
labs/bananadb/trunk/pom.xml
labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/hashtable/Hashtable.java
labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java
Added: labs/bananadb/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/CHANGES.txt?rev=753638&view=auto
==============================================================================
--- labs/bananadb/trunk/CHANGES.txt (added)
+++ labs/bananadb/trunk/CHANGES.txt Sat Mar 14 11:36:58 2009
@@ -0,0 +1,67 @@
+BananaDB Change Log
+
+======================= Trunk (not yet released) =======================
+
+Changes in runtime behavior
+
+
+API Changes
+
+
+File format
+
+ 1. Hashtable and Key postings 1024 bytes headers.
+ (Karl Wettin)
+
+ Th headers contains data start offset (and more) that is read at every
+ access and thus made the way for rehashing on one JVM to "notify" other
+ JVMs to start accessing rehashed data.
+
+
+Bug fixes
+
+
+New features
+
+ 1. Rehashing, growing hashtable capacity.
+ (Karl Wettin)
+
+ Hashtable and Key postings file now both got a header of 1024 bytes
+ that contains start and end offset. On rehash a write lock is set
+ and the hashtable and key postings files are appended with null data
+ on which the rehashed postings are written.
+
+ When rehash finish, the start- and end offset in the file headers are updated
+ and thus any JVM running using the DB will start reading from the rehashed data.
+
+ In the future one should implement
+ 1. A cleanup() method that removes old posting parts of the files.
+ 2. Automatic rehasing when the next available offset hits a threadshold.
+
+
+ CAVEAT!
+ Cursors iterating values will throw an exception in case there has been a rehash.
+
+
+Optimizations
+
+
+Documentation
+
+
+Build
+
+
+Test cases
+
+
+
+
+======================= Version 0.1, 2009-03-14 =======================
+
+First post-snapshot version.
+
+Contains a thread safe, non transactional, file system persistent Hashtable<K, V>.
+(Karl Wettin)
+
+Uses the Lucene LockFactory mechanism.
\ No newline at end of file
Modified: labs/bananadb/trunk/FILES.txt
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/FILES.txt?rev=753638&r1=753637&r2=753638&view=diff
==============================================================================
--- labs/bananadb/trunk/FILES.txt (original)
+++ labs/bananadb/trunk/FILES.txt Sat Mar 14 11:36:58 2009
@@ -5,6 +5,14 @@
Hashtable entity values, the serialized V part of the Hashtable<K, V>.
There can be any number of these files, refered to as partitions in the code.
+Header: 1024 bytes
+{{{
+
+int postings start offset -- will change after rehash
+int capacity
+
+}}}
+
Posting data:
{{{
@@ -20,12 +28,13 @@
or if the capacity is not great enough for keys to get a unique posting the the hashtable
entities will be added to a list that must be iterated in order to find the correct posting.
-This file contains metadata in the first 1024 bytes. This really ought to be factored out to a new file.
-Currently it only contains two values:
+Header: 1024 bytes
{{{
-int next available entry offset
+int next available key posting offset
int entity count
+int postings start offset -- will change after rehash
+int postings end offset -- will change after rehash
}}}
Modified: labs/bananadb/trunk/README.txt
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/README.txt?rev=753638&r1=753637&r2=753638&view=diff
==============================================================================
--- labs/bananadb/trunk/README.txt (original)
+++ labs/bananadb/trunk/README.txt Sat Mar 14 11:36:58 2009
@@ -1,4 +1,6 @@
-This is BananaDB, a Hashtable<K, V> located on filesystem and *not* in memory.
+This is BananaDB, a thread safe and growable Hashtable<K, V>
+located on filesystem and *not* in memory.
+
The aim is to add transactions, secondary indices and a top level annotation API.
An abstract API for this is already available in the source folder,
Modified: labs/bananadb/trunk/TODO.txt
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/TODO.txt?rev=753638&r1=753637&r2=753638&view=diff
==============================================================================
--- labs/bananadb/trunk/TODO.txt (original)
+++ labs/bananadb/trunk/TODO.txt Sat Mar 14 11:36:58 2009
@@ -1,17 +1,5 @@
= Hashtable =
-== Write lock mechanism ==
-
-File write lock for hashtable file, entities file, and one per partition file.
-
-Would it be possible to lock a specific area of the file for writing so that multiple accessors
-could write at the same time but at different places?
-
-== Growing hashtable file ==
-
-Currently the capacity of a hashtable is static. Should this be growable? Or should one combine multiple hashfiles
-using a technique similar to the Lucene MultiReader and merge/optimization?
-
= Transactions =
= Annotation API =
Modified: labs/bananadb/trunk/pom.xml
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/pom.xml?rev=753638&r1=753637&r2=753638&view=diff
==============================================================================
--- labs/bananadb/trunk/pom.xml (original)
+++ labs/bananadb/trunk/pom.xml Sat Mar 14 11:36:58 2009
@@ -1,46 +1,60 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
- <modelVersion>4.0.0</modelVersion>
- <groupId>org.apache.labs</groupId>
- <artifactId>bananadb</artifactId>
- <packaging>jar</packaging>
- <version>0.2-SNAPSHOT</version>
- <name>BananaDB</name>
- <description>BananaDB, a hashtable living on filesystem and not in memory.</description>
- <url>http://kodapan.se/bananadb</url>
-
- <dependencies>
-
- <dependency>
- <groupId>junit</groupId>
- <artifactId>junit</artifactId>
- <version>4.5</version>
- <scope>test</scope>
- </dependency>
-
- <dependency>
- <groupId>commons-logging</groupId>
- <artifactId>commons-logging</artifactId>
- <version>1.1.1</version>
- </dependency>
-
- </dependencies>
- <build>
- <finalName>bananadb</finalName>
-
- <plugins>
-
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-compiler-plugin</artifactId>
- <version>2.0.2</version>
- <configuration>
- <source>1.5</source>
- <target>1.5</target>
- </configuration>
- </plugin>
+ <modelVersion>4.0.0</modelVersion>
+ <groupId>org.apache.labs</groupId>
+ <artifactId>bananadb</artifactId>
+ <packaging>jar</packaging>
+ <version>0.2-SNAPSHOT</version>
+ <name>BananaDB</name>
+ <description>BananaDB, a hashtable living on filesystem and not in memory.</description>
+ <url>http://kodapan.se/bananadb</url>
+
+ <dependencies>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>4.5</version>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ <version>1.4</version>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>log4j</groupId>
+ <artifactId>log4j</artifactId>
+ <version>1.2.15</version>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>commons-logging</groupId>
+ <artifactId>commons-logging</artifactId>
+ <version>1.1.1</version>
+ </dependency>
+
+ </dependencies>
+ <build>
+ <finalName>bananadb</finalName>
+
+ <plugins>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>2.0.2</version>
+ <configuration>
+ <source>1.5</source>
+ <target>1.5</target>
+ </configuration>
+ </plugin>
- </plugins>
+ </plugins>
- </build>
+ </build>
</project>
Modified: labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/hashtable/Hashtable.java
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/hashtable/Hashtable.java?rev=753638&r1=753637&r2=753638&view=diff
==============================================================================
--- labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/hashtable/Hashtable.java (original)
+++ labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/hashtable/Hashtable.java Sat Mar 14 11:36:58 2009
@@ -31,21 +31,20 @@
* Hashtable<K, V> that lives on local disk via random access file.
* <p/>
* It does not handle null keys but allows null values.
- * It is limited to a maximum number of entities, defined when opening (formatting) the map.
* <p/>
* todo: optimize
- * todo: rehash()
- * todo: use partitioned space for entries and not just for value data
* todo: overwrite old value in data file if possible when replacing a value. keep track of unused positions
*/
public class Hashtable<K, V> {
private Log log = LogFactory.getLog(Hashtable.class);
+ private static final int HASHTABLE_HEADER_BYTE_SIZE = 1024;
+
private static final int HASHTABLE_RECORD_BYTE_SIZE = 1 + 4;
/**
- * the header contains meta data such as number of records, et c. todo factor out to new meta data file
+ * the header contains meta data such as number of records, et c.
*/
private static final int KEY_POSTINGS_HEADER_BYTE_SIZE = 1024;
@@ -71,29 +70,12 @@
private ValueClassHandler<V> valueClassHandler;
-
- private int capacity;
- private int entityCount;
- private int nextKeyPostingOffset;
-
private long lockWaitTimeoutMilliseconds = 10000;
private Lock writeLock;
- private Lock readLock;
-
-
- private void setNextKeyPostingOffset(int nextKeyPostingOffset) {
- this.nextKeyPostingOffset = nextKeyPostingOffset;
- }
-
- private int getNextKeyPostingOffset() {
- return nextKeyPostingOffset;
- }
-
-// // todo factor out RAF to new reader (transaction) class.
-// private RandomAccessFile hashtableRAF;
-// private RandomAccessFile entriesRAF;
-
+ /**
+ * todo Hashtable should not be aware of these! they should detect new partitions automatically
+ */
private List<HashtableAccessor> accessors = new ArrayList<HashtableAccessor>();
/**
@@ -123,15 +105,14 @@
* Milage might vary depending on distribution of key value hash codes.
*
* @param path Directory where hashtable data is stored. Only one hashtable can fit in this directory.
- * @param capacity Maximum number of entites that fits this hashtable. This value is only valid when creating a new hashtable.
+ * @param initalCapacity Maximum number of entites that fits this hashtable. This value is only valid when creating a new hashtable.
* @param keyClassHandler
* @param valueClassHandler
* @throws IOException
*/
- public Hashtable(File path, int capacity, KeyClassHandler<K> keyClassHandler, ValueClassHandler<V> valueClassHandler, LockFactory lockFactory) throws IOException {
+ public Hashtable(final File path, final int initalCapacity, final KeyClassHandler<K> keyClassHandler, final ValueClassHandler<V> valueClassHandler, LockFactory lockFactory) throws IOException {
writeLock = lockFactory.makeLock("write");
- readLock = lockFactory.makeLock("read");
this.keyClassHandler = keyClassHandler;
this.valueClassHandler = valueClassHandler;
@@ -147,53 +128,75 @@
hashtableFile = new File(path, hashtableFileName);
this.keyPostingsFile = new File(path, keyPostingsFileName);
- // open
-
- File hashtableFile = new File(path, hashtableFileName);
+ boolean open;
if (!hashtableFile.exists()) {
- format(hashtableFile, capacity * HASHTABLE_RECORD_BYTE_SIZE);
- } else {
- capacity = (int) (hashtableFile.length() / HASHTABLE_RECORD_BYTE_SIZE);
- }
- this.capacity = capacity;
+ // attempt to create filesystem
- File keyPostingsFile = new File(path, keyPostingsFileName);
- if (!keyPostingsFile.exists()) {
+ Lock.With<Boolean> with = new Lock.With<Boolean>(writeLock, 60000) {
+ protected Boolean doBody() throws IOException {
- log.info("Creating a new key postings file");
+ if (hashtableFile.exists()) {
+ return false;
+ }
- format(keyPostingsFile, KEY_POSTINGS_HEADER_BYTE_SIZE + (capacity * keyPostingByteSize));
- setNextKeyPostingOffset(KEY_POSTINGS_HEADER_BYTE_SIZE);
+ log.info("Creating hashtable file");
- RandomAccessFile keyPostingsRAF = new RandomAccessFile(keyPostingsFile, "rw");
- keyPostingsRAF.seek(0);
- keyPostingsRAF.writeInt(getNextKeyPostingOffset());
- keyPostingsRAF.close();
+ File hashtableFile = new File(path, hashtableFileName);
+ format(hashtableFile, HASHTABLE_HEADER_BYTE_SIZE + (initalCapacity * HASHTABLE_RECORD_BYTE_SIZE));
- } else {
+ HashtableFileHeader hashtableFileHeader = new HashtableFileHeader(HASHTABLE_HEADER_BYTE_SIZE, initalCapacity);
+ RandomAccessFile raf = new RandomAccessFile(hashtableFile, "rw");
+ hashtableFileHeader.save(raf);
+ raf.close();
- RandomAccessFile keyPostingsRAF = new RandomAccessFile(keyPostingsFile, "r");
- keyPostingsRAF.seek(0);
- setNextKeyPostingOffset(keyPostingsRAF.readInt());
- entityCount = keyPostingsRAF.readInt();
- keyPostingsRAF.close();
- }
+ log.info("Creating key postings file");
- File[] valueFiles = path.listFiles(new FileFilter() {
- public boolean accept(File file) {
- return file.isFile() && file.getName().matches(valuePostingsFileName + "\\.[0-9]+");
- }
- });
- Arrays.sort(valueFiles, new Comparator<File>() {
- public int compare(File file, File file1) {
- return file.getName().compareTo(file1.getName());
+ File keyPostingsFile = new File(path, keyPostingsFileName);
+ format(keyPostingsFile, KEY_POSTINGS_HEADER_BYTE_SIZE + (initalCapacity * keyPostingByteSize));
+
+ KeyPostingsFileHeader keyPostingsFileHeader = new KeyPostingsFileHeader(KEY_POSTINGS_HEADER_BYTE_SIZE, 0, KEY_POSTINGS_HEADER_BYTE_SIZE, initalCapacity);
+ raf = new RandomAccessFile(keyPostingsFile, "rw");
+ keyPostingsFileHeader.save(raf);
+ raf.close();
+
+
+ log.info("Creating first value partition file");
+ Partition partition = new Partition();
+ partition.format(partitionSize);
+ partition.open();
+
+ return true;
+ }
+ };
+
+ if (!with.run()) {
+ log.warn("Entered method to create filesystem but another process already created it.");
+ open = true;
+ } else {
+ open = false;
}
- });
- System.currentTimeMillis(); // todo check order
- if (valueFiles.length > 0) {
+ } else {
+ open = true;
+ }
+
+ if (open) {
+
+ // open
+
+ File[] valueFiles = path.listFiles(new FileFilter() {
+ public boolean accept(File file) {
+ return file.isFile() && file.getName().matches(valuePostingsFileName + "\\.[0-9]+");
+ }
+ });
+ Arrays.sort(valueFiles, new Comparator<File>() {
+ public int compare(File file, File file1) {
+ return file.getName().compareTo(file1.getName());
+ }
+ });
+
for (int i = 0; i < valueFiles.length; i++) {
if (!valueFiles[i].getName().endsWith("." + i)) {
throw new FileNotFoundException("Expected partition " + valuePostingsFileName + "." + i + " but found " + valueFiles[i].getName());
@@ -201,15 +204,171 @@
Partition partition = new Partition(i);
partition.open();
}
- } else {
- Partition partition = new Partition();
- partition.format(partitionSize);
- partition.open();
+ }
+
+
+ }
+
+ public class HashtableFileHeader {
+ private int startOffset;
+ private int capacity;
+
+ private HashtableFileHeader(int startOffset, int capacity) {
+ this.startOffset = startOffset;
+ this.capacity = capacity;
+ }
+
+ private HashtableFileHeader(HashtableAccessor accessor) throws IOException {
+ load(accessor);
+ }
+
+ private void load(HashtableAccessor accessor) throws IOException {
+ accessor.getHashtableRAF().seek(0);
+ startOffset = accessor.getHashtableRAF().readInt();
+ capacity = accessor.getHashtableRAF().readInt();
+ }
+
+ private void save(HashtableAccessor accessor) throws IOException {
+ save(accessor.getHashtableRAF());
+ }
+
+ public int getHashCodeTableOffset(int hashCode) {
+ return startOffset + (HASHTABLE_RECORD_BYTE_SIZE * (hashCode & (capacity - 1)));
+ }
+ private void save(RandomAccessFile raf) throws IOException {
+ raf.seek(0);
+ raf.writeInt(startOffset);
+ raf.writeInt(capacity);
}
+ public int getStartOffset() {
+ return startOffset;
+ }
+
+ public int getCapacity() {
+ return capacity;
+ }
+
+ public int getEndOffset() {
+ return startOffset + (capacity * HASHTABLE_RECORD_BYTE_SIZE);
+ }
+
+ private void setStartOffset(int startOffset) {
+ this.startOffset = startOffset;
+ }
+
+ private void setCapacity(int capacity) {
+ this.capacity = capacity;
+ }
+ @Override
+ public String toString() {
+ return "HashtableFileHeader{" +
+ "startOffset=" + startOffset +
+ ", capacity=" + capacity +
+ '}';
+ }
+ }
+
+ public class KeyPostingsFileHeader {
+ private int nextOffset;
+ private int entityCount;
+ private int startOffset;
+ private int capacity;
+
+ public KeyPostingsFileHeader(int nextOffset, int entityCount, int startOffset, int capacity) {
+ this.nextOffset = nextOffset;
+ this.entityCount = entityCount;
+ this.startOffset = startOffset;
+ this.capacity = capacity;
+ }
+
+ private KeyPostingsFileHeader(HashtableAccessor accessor) throws IOException {
+ load(accessor);
+ }
+
+ private void load(HashtableAccessor accessor) throws IOException {
+ accessor.getKeyPostingsRAF().seek(0);
+ nextOffset = accessor.getKeyPostingsRAF().readInt();
+ entityCount = accessor.getKeyPostingsRAF().readInt();
+ startOffset = accessor.getKeyPostingsRAF().readInt();
+ capacity = accessor.getKeyPostingsRAF().readInt();
+ }
+
+ private void save(HashtableAccessor accessor) throws IOException {
+ save(accessor.getKeyPostingsRAF());
+ }
+
+ private void save(RandomAccessFile raf) throws IOException {
+ raf.seek(0);
+ raf.writeInt(nextOffset);
+ raf.writeInt(entityCount);
+ raf.writeInt(startOffset);
+ raf.writeInt(capacity);
+ }
+
+ public int getNextOffset() {
+ return nextOffset;
+ }
+
+ public int getEntityCount() {
+ return entityCount;
+ }
+
+ public int getStartOffset() {
+ return startOffset;
+ }
+
+ public int getCapacity() {
+ return capacity;
+ }
+
+ public int getEndOffset() {
+ return startOffset + (capacity * keyPostingByteSize);
+ }
+
+ private int increaseNextOffset(int bytes) {
+ nextOffset += bytes;
+ return nextOffset;
+ }
+
+ private void setNextOffset(int nextOffset) {
+ this.nextOffset = nextOffset;
+ }
+
+ private int increaseEntityCount(int items) {
+ entityCount += items;
+ return entityCount;
+ }
+
+ private int decreaseEntityCount(int items) {
+ entityCount -= items;
+ return entityCount;
+ }
+
+ private void setEntityCount(int entityCount) {
+ this.entityCount = entityCount;
+ }
+
+ private void setStartOffset(int startOffset) {
+ this.startOffset = startOffset;
+ }
+
+ private void setCapacity(int capacity) {
+ this.capacity = capacity;
+ }
+
+ @Override
+ public String toString() {
+ return "KeyPostingsFileHeader{" +
+ "nextOffset=" + nextOffset +
+ ", entityCount=" + entityCount +
+ ", startOffset=" + startOffset +
+ ", capacity=" + capacity +
+ '}';
+ }
}
/**
@@ -233,6 +392,8 @@
}
/**
+ * EXPERT: This method should only be called when a lock exists.
+ *
* @param accessor
* @param key
* @param value
@@ -241,6 +402,9 @@
*/
private V doPut(HashtableAccessor accessor, K key, V value) throws IOException {
+ HashtableFileHeader hashtableFileHeader = new HashtableFileHeader(accessor);
+ KeyPostingsFileHeader keyPostingsFileHeader = new KeyPostingsFileHeader(accessor);
+
// write key to buf
ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);
keyClassHandler.write(key, new DataOutputStream(baos));
@@ -260,23 +424,22 @@
V oldValue = null;
int keyHash = key.hashCode();
- int index = keyHash & (capacity - 1);
- byte[] hashtableRecordBuf = new byte[HASHTABLE_RECORD_BYTE_SIZE];
- accessor.getHashtableRAF().seek(index * HASHTABLE_RECORD_BYTE_SIZE);
+ int hashtableOffset = hashtableFileHeader.getHashCodeTableOffset(keyHash);
+
+ accessor.getHashtableRAF().seek(hashtableOffset);
if (accessor.getHashtableRAF().readByte() == 0) {
// create inital posting for hash
- int keyPostingOffset = getNextKeyPostingOffset();
- setNextKeyPostingOffset(getNextKeyPostingOffset() + keyPostingByteSize);
- accessor.getKeyPostingsRAF().seek(0);
- accessor.getKeyPostingsRAF().writeInt(getNextKeyPostingOffset());
- accessor.getKeyPostingsRAF().writeInt(++entityCount);
+ int keyPostingOffset = keyPostingsFileHeader.getNextOffset();
+ keyPostingsFileHeader.increaseNextOffset(keyPostingByteSize);
+ keyPostingsFileHeader.increaseEntityCount(1);
+ keyPostingsFileHeader.save(accessor);
accessor.getKeyPostingsRAF().seek(keyPostingOffset);
writePostings(accessor, -1, keyBuf, valueBuf, keyHash);
- accessor.getHashtableRAF().seek(index * HASHTABLE_RECORD_BYTE_SIZE);
+ accessor.getHashtableRAF().seek(hashtableOffset);
accessor.getHashtableRAF().writeByte(1);
accessor.getHashtableRAF().writeInt(keyPostingOffset);
@@ -327,11 +490,13 @@
}
// this key instance did not exist in entries
- // although it shares hash with some other keys in there.
+ // although it shares hash code or hashtable position with some other keys in there.
// create new key posting and point at it from the previous key posting
- int keyPostingOffset = getNextKeyPostingOffset();
- setNextKeyPostingOffset(getNextKeyPostingOffset() + keyPostingByteSize);
+ int keyPostingOffset = keyPostingsFileHeader.getNextOffset();
+ keyPostingsFileHeader.increaseNextOffset(keyPostingByteSize);
+ keyPostingsFileHeader.increaseEntityCount(1);
+ keyPostingsFileHeader.save(accessor);
accessor.getKeyPostingsRAF().seek(keyPostingOffset);
writePostings(accessor, -1, keyBuf, valueBuf, keyHash);
@@ -339,9 +504,6 @@
accessor.getKeyPostingsRAF().seek(previousKeyPostingOffset);
accessor.getKeyPostingsRAF().writeInt(keyPostingOffset);
- accessor.getKeyPostingsRAF().seek(4);
- accessor.getKeyPostingsRAF().writeInt(++entityCount);
-
return null;
}
@@ -376,18 +538,14 @@
// int offset position in value partition, -1 == null
accessor.getKeyPostingsRAF().writeInt((int) offset);
- // int key length in bytes, 0 = null
- if (keyBuf == null || keyBuf.length == 0) {
- accessor.getKeyPostingsRAF().writeInt(0);
- } else {
- accessor.getKeyPostingsRAF().writeInt(keyBuf.length);
+ // int key byte length todo: remove this from the file
+ accessor.getKeyPostingsRAF().writeInt(keyBuf.length);
- // key writable key
- accessor.getKeyPostingsRAF().write(keyBuf);
+ // key writable key
+ accessor.getKeyPostingsRAF().write(keyBuf);
- // key hash
- accessor.getKeyPostingsRAF().writeInt(keyHashCode);
- }
+ // key hash
+ accessor.getKeyPostingsRAF().writeInt(keyHashCode);
}
@@ -398,10 +556,15 @@
* @throws IOException
*/
public boolean containsKey(HashtableAccessor accessor, K key) throws IOException {
+
+ HashtableFileHeader hashtableFileHeader = new HashtableFileHeader(accessor);
+ KeyPostingsFileHeader keyPostingsFileHeader = new KeyPostingsFileHeader(accessor);
+
int keyHash = key.hashCode();
- int index = keyHash & (capacity - 1);
+ int hashtableOffset = hashtableFileHeader.getHashCodeTableOffset(keyHash);
+
byte[] hashtableRecordBuf = new byte[HASHTABLE_RECORD_BYTE_SIZE];
- accessor.getHashtableRAF().seek(index * HASHTABLE_RECORD_BYTE_SIZE);
+ accessor.getHashtableRAF().seek(hashtableOffset);
accessor.getHashtableRAF().read(hashtableRecordBuf);
DataInput in = new DataInputStream(new ByteArrayInputStream(hashtableRecordBuf));
byte isnull = in.readByte();
@@ -425,10 +588,6 @@
nextKeyPostingOffset = accessor.getKeyPostingsRAF().readInt();
int partitionNumber = accessor.getKeyPostingsRAF().readInt();
-
-// if (partitionNumber == -1) {
-// return false;
-// }
int partitionOffset = accessor.getKeyPostingsRAF().readInt();
int keySize = accessor.getKeyPostingsRAF().readInt();
@@ -447,18 +606,19 @@
public V get(HashtableAccessor accessor, K key) throws IOException {
+ HashtableFileHeader hashtableFileHeader = new HashtableFileHeader(accessor);
+ KeyPostingsFileHeader keyPostingsFileHeader = new KeyPostingsFileHeader(accessor);
+
int keyHash = key.hashCode();
- int index = keyHash & (capacity - 1);
- byte[] hashtableRecordBuf = new byte[HASHTABLE_RECORD_BYTE_SIZE];
- accessor.getHashtableRAF().seek(index * HASHTABLE_RECORD_BYTE_SIZE);
- accessor.getHashtableRAF().read(hashtableRecordBuf);
- DataInput in = new DataInputStream(new ByteArrayInputStream(hashtableRecordBuf));
- byte isnull = in.readByte();
+ int hashtableOffset = hashtableFileHeader.getHashCodeTableOffset(keyHash);
+ accessor.getHashtableRAF().seek(hashtableOffset);
+
+ byte isnull = accessor.getHashtableRAF().readByte();
if (isnull == 0) {
return null;
}
- int firstKeyPostingOffset = in.readInt();
+ int firstKeyPostingOffset = accessor.getHashtableRAF().readInt();
K compareKey;
@@ -500,7 +660,16 @@
return null;
}
+
+ /**
+ * todo doRemove and lock in remove.
+ */
public V remove(HashtableAccessor accessor, K key) throws IOException {
+
+ HashtableFileHeader hashtableFileHeader = new HashtableFileHeader(accessor);
+ KeyPostingsFileHeader keyPostingsFileHeader = new KeyPostingsFileHeader(accessor);
+
+
// write key to buf
ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);
keyClassHandler.write(key, new DataOutputStream(baos));
@@ -508,15 +677,12 @@
int keyHash = key.hashCode();
- int index = keyHash & (capacity - 1);
- byte[] hashtableRecordBuf = new byte[HASHTABLE_RECORD_BYTE_SIZE];
- accessor.getHashtableRAF().seek(index * HASHTABLE_RECORD_BYTE_SIZE);
- accessor.getHashtableRAF().read(hashtableRecordBuf);
- DataInput in = new DataInputStream(new ByteArrayInputStream(hashtableRecordBuf));
+ int hashtableOffset = hashtableFileHeader.getHashCodeTableOffset(keyHash);
+ accessor.getHashtableRAF().seek(hashtableOffset);
- byte isnull = in.readByte();
+ byte isnull = accessor.getHashtableRAF().readByte();
- int firstKeyPostingOffset = in.readInt();
+ int firstKeyPostingOffset = accessor.getHashtableRAF().readInt();
K compareKey;
@@ -552,12 +718,12 @@
accessor.getKeyPostingsRAF().writeInt(nextKeyPostingOffset);
} else {
// remove posting in hashtable
- accessor.getHashtableRAF().seek(index * HASHTABLE_RECORD_BYTE_SIZE);
+ accessor.getHashtableRAF().seek(hashtableOffset);
accessor.getHashtableRAF().writeByte(0);
}
- accessor.getKeyPostingsRAF().seek(4);
- accessor.getKeyPostingsRAF().writeInt(--entityCount);
+ keyPostingsFileHeader.decreaseEntityCount(1);
+ keyPostingsFileHeader.save(accessor);
return oldValue;
}
@@ -570,40 +736,54 @@
throw new RuntimeException("Expected to find the value in data file!");
}
- public void rehash(final int capacity) throws IOException {
- if (writeLock == null) {
- doRehash(capacity);
- } else {
- Lock.With<V> with = new Lock.With<V>(writeLock, lockWaitTimeoutMilliseconds) {
- protected V doBody() throws IOException {
- doRehash(capacity);
- return null;
- }
- };
- with.run();
- }
+ /**
+ * Rehashing (allows for growing hashtable capacity)
+ *
+ * The hashtable and key postings files are appended with null data
+ * on which the rehashed postings are written.
+ *
+ * When rehash finish the start and end offset in the file headers are updated
+ * and thus any JVM running using the DB will start reading from the rehashed data.
+ *
+ * This will require a read and write locking optimize() method in the future.
+ *
+ * Cursors iterating values will throw an exception in case there has been a rehash.
+ *
+ * @param accessor
+ * @param capacity
+ * @throws IOException
+ */
+ public void rehash(final HashtableAccessor accessor, final int capacity) throws IOException {
+ Lock.With<V> with = new Lock.With<V>(writeLock, lockWaitTimeoutMilliseconds) {
+ protected V doBody() throws IOException {
+ doRehash(accessor, capacity);
+ return null;
+ }
+ };
+ with.run();
}
- private void doRehash(final int capacity) throws IOException {
- final File hashtableFile = new File(path, "hashtable.rehash");
- format(hashtableFile, capacity * HASHTABLE_RECORD_BYTE_SIZE);
+ private void doRehash(HashtableAccessor accessor, int newCapacity) throws IOException {
- final File keyPostingsFile = new File(path, "keys.rehash");
- format(keyPostingsFile, KEY_POSTINGS_HEADER_BYTE_SIZE + (capacity * keyPostingByteSize));
+ // todo append to existing files rather than creating new!
- RandomAccessFile hashtableOutput = new RandomAccessFile(hashtableFile, "rw");
+ HashtableFileHeader inputHashtableFileHeader = new HashtableFileHeader(accessor);
+ KeyPostingsFileHeader inputKeyPostingsFileHeader = new KeyPostingsFileHeader(accessor);
- int read;
+ HashtableFileHeader outputHashtableFileHeader = new HashtableFileHeader(inputHashtableFileHeader.getEndOffset() + 1, newCapacity);
+ KeyPostingsFileHeader outputKeyPostingsFileHeader = new KeyPostingsFileHeader(inputKeyPostingsFileHeader.getNextOffset() + 1, inputKeyPostingsFileHeader.getEntityCount(), inputKeyPostingsFileHeader.getNextOffset() + 1, newCapacity);
- RandomAccessFile keyPostingsInput = new RandomAccessFile(this.keyPostingsFile, "r");
- RandomAccessFile keyPostingsOutput = new RandomAccessFile(keyPostingsFile, "rw");
+ RandomAccessFile keyPostingsInput = accessor.getKeyPostingsRAF();
+ keyPostingsInput.seek(inputKeyPostingsFileHeader.getStartOffset());
+
+ HashtableAccessor outputAccessor = createAccessor(false);
+ RandomAccessFile hashtableOutput = outputAccessor.getHashtableRAF();
+ RandomAccessFile keyPostingsOutput = outputAccessor.getKeyPostingsRAF();
+
+ // append formatted data to files
+ formatAppend(hashtableOutput, newCapacity * HASHTABLE_RECORD_BYTE_SIZE);
+ formatAppend(keyPostingsOutput, newCapacity * keyPostingByteSize);
- byte[] buf = new byte[KEY_POSTINGS_HEADER_BYTE_SIZE];
- read = keyPostingsInput.read(buf);
- if (read != KEY_POSTINGS_HEADER_BYTE_SIZE) {
- throw new IOException("Expected " + KEY_POSTINGS_HEADER_BYTE_SIZE + " bytes long key postings header");
- }
- keyPostingsOutput.write(buf);
int nextKeyListPostingOffset;
int valuePartition;
@@ -612,9 +792,7 @@
byte[] keyBuf = new byte[keyClassHandler.getByteSize()];
int keyHash;
- int nextAvailableKeyPostingOffset = KEY_POSTINGS_HEADER_BYTE_SIZE;
-
- for (int inputKeyPostingIndex = 0; inputKeyPostingIndex < size(); inputKeyPostingIndex++) {
+ for (int inputKeyPostingIndex = 0; inputKeyPostingIndex < inputKeyPostingsFileHeader.getEntityCount(); inputKeyPostingIndex++) {
// int offset position in this file to the next entry with the same hash, -1 = null
@@ -631,15 +809,15 @@
keyHash = keyPostingsInput.readInt();
- int index = keyHash & (capacity - 1);
- hashtableOutput.seek(index * HASHTABLE_RECORD_BYTE_SIZE);
+ int outputHashtableOffset = outputHashtableFileHeader.getHashCodeTableOffset(keyHash);
+ hashtableOutput.seek(outputHashtableOffset);
if (hashtableOutput.readByte() == 0) {
// create inital posting for hash
- int keyPostingOffset = nextAvailableKeyPostingOffset;
- nextAvailableKeyPostingOffset += keyPostingByteSize;
+ int outputKeyPostingOffset = outputKeyPostingsFileHeader.getNextOffset();
+ outputKeyPostingsFileHeader.increaseNextOffset(keyPostingByteSize);
- keyPostingsOutput.seek(keyPostingOffset);
+ keyPostingsOutput.seek(outputKeyPostingOffset);
// int offset position in this file to the next key posting with the same hash, -1 = null
keyPostingsOutput.writeInt(-1);
@@ -649,22 +827,18 @@
// int offset position in value partition, -1 == null
keyPostingsOutput.writeInt((int) valuePartitionOffset);
- // int key length in bytes, 0 = null
- if (keyBuf == null || keyBuf.length == 0) {
- keyPostingsOutput.writeInt(0);
- } else {
- keyPostingsOutput.writeInt(keyBuf.length);
+ // int key length in bytes, 0 = null // todo remove
+ keyPostingsOutput.writeInt(keyBuf.length);
- // key writable key
- keyPostingsOutput.write(keyBuf);
+ // key writable key
+ keyPostingsOutput.write(keyBuf);
- // key hash
- keyPostingsOutput.write(keyHash);
- }
+ // key hash
+ keyPostingsOutput.write(keyHash);
- hashtableOutput.seek(index * HASHTABLE_RECORD_BYTE_SIZE);
+ hashtableOutput.seek(outputHashtableOffset);
hashtableOutput.writeByte(1);
- hashtableOutput.writeInt(keyPostingOffset);
+ hashtableOutput.writeInt(outputKeyPostingOffset);
} else {
// already contains a posting for the hash
@@ -692,8 +866,8 @@
// although it shares hash with some other keys in there.
// create new key posting and point at it from the previous key posting
- int keyPostingOffset = nextAvailableKeyPostingOffset;
- nextAvailableKeyPostingOffset += keyPostingByteSize;
+ int keyPostingOffset = outputKeyPostingsFileHeader.getNextOffset();
+ outputKeyPostingsFileHeader.increaseNextOffset(keyPostingByteSize);
keyPostingsOutput.seek(keyPostingOffset);
// int offset position in this file to the next key posting with the same hash, -1 = null
@@ -705,18 +879,14 @@
// int offset position in value partition, -1 == null
keyPostingsOutput.writeInt((int) valuePartitionOffset);
- // int key length in bytes, 0 = null
- if (keyBuf == null || keyBuf.length == 0) {
- keyPostingsOutput.writeInt(0);
- } else {
- keyPostingsOutput.writeInt(keyBuf.length);
+ // int key length in bytes, 0 = null // todo remove
+ keyPostingsOutput.writeInt(keyBuf.length);
- // key writable key
- keyPostingsOutput.write(keyBuf);
+ // key writable key
+ keyPostingsOutput.write(keyBuf);
- // key hash
- keyPostingsOutput.write(keyHash);
- }
+ // key hash
+ keyPostingsOutput.write(keyHash);
keyPostingsOutput.seek(previousKeyPostingOffset);
keyPostingsOutput.writeInt(keyPostingOffset);
@@ -724,32 +894,12 @@
}
}
- keyPostingsInput.close();
- keyPostingsOutput.close();
-
- hashtableOutput.close();
-
- Lock.With with = new Lock.With(readLock, lockWaitTimeoutMilliseconds) {
- protected Object doBody() throws IOException {
-
- // lock all accessors for reading
-
- Hashtable.this.hashtableFile.delete();
- hashtableFile.renameTo(Hashtable.this.hashtableFile);
-
- Hashtable.this.keyPostingsFile.delete();
- keyPostingsFile.renameTo(Hashtable.this.keyPostingsFile);
+ outputAccessor.close();
- Hashtable.this.capacity = capacity;
+ // write new start offset and capacity to hashtable and keypostings files
+ outputHashtableFileHeader.save(accessor);
+ outputKeyPostingsFileHeader.save(accessor);
- // todo reopen all accessor RAFs of hashtable and keys file
-
- // unlock all accessors for reading
-
- return null;
- }
- };
- with.run();
System.currentTimeMillis();
@@ -869,9 +1019,34 @@
}
+ public void formatAppend(RandomAccessFile raf, long size) throws IOException {
- public int size() {
- return entityCount;
+ log.info("Appending " + size + " bytes..");
+
+ long ms = System.currentTimeMillis();
+
+ long leftToWrite = size;
+
+ raf.seek(raf.length());
+
+ byte[] bytes = new byte[32 * 1024];
+ while (leftToWrite >= bytes.length) {
+ raf.write(bytes);
+ leftToWrite -= bytes.length;
+ }
+ if (leftToWrite > 0) {
+ raf.write(bytes, 0, (int) leftToWrite);
+ }
+
+
+ log.info("It took " + (System.currentTimeMillis() - ms) + " milliseconds to append.");
+
+ }
+
+
+ public int size(HashtableAccessor accessor) throws IOException {
+ KeyPostingsFileHeader keyPostingsFileHeader = new KeyPostingsFileHeader(accessor);
+ return keyPostingsFileHeader.getEntityCount();
}
@@ -975,7 +1150,7 @@
this.accessor = accessor;
}
- private long hashtableCursor = 0;
+ private long hashtableCursor = -1;
private long keyPostingsCursor = -1;
private K key;
@@ -988,6 +1163,11 @@
* @throws IOException
*/
public boolean next() throws IOException {
+ if (hashtableCursor == -1) {
+ HashtableFileHeader hashtableFileHeader = new HashtableFileHeader(accessor);
+ hashtableCursor = hashtableFileHeader.getStartOffset();
+ }
+
if (keyPostingsCursor != -1) {
accessor.getKeyPostingsRAF().seek(keyPostingsCursor);
keyPostingsCursor = accessor.getKeyPostingsRAF().readInt(); // next entity
@@ -997,7 +1177,14 @@
return true;
}
}
- while (hashtableCursor < (capacity * HASHTABLE_RECORD_BYTE_SIZE)) {
+
+ HashtableFileHeader hashtableFileHeader = new HashtableFileHeader(accessor);
+ if (hashtableFileHeader.getStartOffset() > hashtableCursor) {
+ // Rehash occured while iterating.
+ throw new IOException("Rehash occured while iterating!"); // todo
+ }
+
+ while (hashtableCursor < (hashtableFileHeader.getEndOffset())) {
accessor.getHashtableRAF().seek(hashtableCursor);
hashtableCursor += HASHTABLE_RECORD_BYTE_SIZE;
byte isnull = accessor.getHashtableRAF().readByte();
Added: labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/index/README.txt
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/index/README.txt?rev=753638&view=auto
==============================================================================
--- labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/index/README.txt (added)
+++ labs/bananadb/trunk/src/main/java/org/apache/labs/bananadb/index/README.txt Sat Mar 14 11:36:58 2009
@@ -0,0 +1 @@
+Implementing the stuff in this package is due by version 1.0
\ No newline at end of file
Added: labs/bananadb/trunk/src/test/java/log4j.properties
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/src/test/java/log4j.properties?rev=753638&view=auto
==============================================================================
--- labs/bananadb/trunk/src/test/java/log4j.properties (added)
+++ labs/bananadb/trunk/src/test/java/log4j.properties Sat Mar 14 11:36:58 2009
@@ -0,0 +1,10 @@
+# ***** Set root logger level to DEBUG and its only appender to A.
+log4j.rootLogger=info, A
+
+# ***** A is set to be a ConsoleAppender.
+log4j.appender.A=org.apache.log4j.ConsoleAppender
+# ***** A uses PatternLayout.
+log4j.appender.A.layout=org.apache.log4j.PatternLayout
+#log4j.appender.A.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
+log4j.appender.A.layout.ConversionPattern=%-5p %c %x - %m%n
+
Added: labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/HashtableTest.java
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/HashtableTest.java?rev=753638&view=auto
==============================================================================
--- labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/HashtableTest.java (added)
+++ labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/HashtableTest.java Sat Mar 14 11:36:58 2009
@@ -0,0 +1,57 @@
+package org.apache.labs.bananadb.hashtable;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import junit.framework.TestCase;
+
+import java.io.File;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.logging.LogFactory;
+import org.apache.commons.logging.Log;
+
+/**
+ * @author kalle
+ * @since 2009-mar-14 11:38:43
+ */
+public abstract class HashtableTest extends TestCase {
+
+ public static final Log log = LogFactory.getLog(HashtableTest.class);
+
+ private File path;
+
+ protected HashtableTest() {
+ path = new File("target/hashtables");
+ if (!path.exists() && !path.mkdirs()) {
+ throw new RuntimeException("Could not create path " + path);
+ }
+ }
+
+ protected File getDirectory(String name) {
+ File path = new File(this.path, name);
+ if (path.exists()) {
+ FileUtils.deleteQuietly(path);
+ }
+ if (!path.mkdirs()) {
+ throw new RuntimeException("Could not create path " + path);
+ }
+ return path;
+ }
+
+
+}
Modified: labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java?rev=753638&r1=753637&r2=753638&view=diff
==============================================================================
--- labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java (original)
+++ labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java Sat Mar 14 11:36:58 2009
@@ -35,162 +35,30 @@
* Date: 2009-jan-03
* Time: 05:43:56
*/
-public class TestHashtable extends TestCase {
+public class TestHashtable extends HashtableTest {
- private File path;
-
- public TestHashtable() {
- path = new File("target/" + Hashtable.class.getSimpleName() + "/" + System.currentTimeMillis());
- if (!path.exists() && !path.mkdirs()) {
- throw new RuntimeException("Could not create path " + path);
- }
- }
-
-
-
-
- @Test
- public void testRehash() throws Exception {
-
- File path = new File(this.path, "testRehash");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
- HashtableAccessor accessor = map.createAccessor(false);
-
- map.put(accessor, 0, "hello world");
- map.put(accessor, 1, "greetings world");
- map.put(accessor, 5, "salutations world");
-
- accessor.close();
-
- map.rehash(1000);
-
- assertEquals(3, map.size());
-
- accessor = map.createAccessor(false);
- assertEquals("hello world", map.get(accessor, 0));
- assertEquals("greetings world", map.get(accessor, 1));
- assertEquals("salutations world", map.get(accessor, 5));
-
- assertEquals(3, map.size());
-
- // todo cursor
-
- accessor.close();
-
- // todo close and open again, then assert the same post-rehash tests once again
-
-
-
- }
-
-
- private class TestThreadedRunnable implements Runnable {
- private Hashtable<Integer, String> map;
- private HashtableAccessor accessor;
- private Exception exception;
- private AtomicInteger pk;
- private int id;
-
- private TestThreadedRunnable(Hashtable<Integer, String> map, AtomicInteger pk) throws IOException {
- this.map = map;
- this.pk = pk;
- accessor = map.createAccessor(false);
- }
-
- public void run() {
- try {
- for (int i = 0; i < 1000; i++) {
- map.put(accessor, pk.getAndIncrement(), "foobar");
- System.out.println(id + "\t" + i);
- }
- } catch (Exception exception) {
- this.exception = exception;
- throw new RuntimeException(exception);
- }
- }
- }
-
- @Test
- public void testThreaded() throws Exception {
- File path = new File(this.path, "testThreaded");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 40000, new IntegerHandler(), new StringValueHandler());
- map.setLockWaitTimeoutMilliseconds(60000);
- AtomicInteger pk = new AtomicInteger(0);
-
- List<Thread> threads = new ArrayList<Thread>();
- List<TestThreadedRunnable> runnables = new ArrayList<TestThreadedRunnable>();
-
- for (int i = 0; i < 10; i++) {
- TestThreadedRunnable runnable = new TestThreadedRunnable(map, pk);
- runnable.id = i;
- runnables.add(runnable);
- Thread thread = new Thread(runnable);
- thread.setName("TestThread " + String.valueOf(i));
- threads.add(thread);
- }
- for (Thread thread : threads) {
- thread.start();
- }
- for (Thread thread : threads) {
- thread.join();
- }
-
- for (TestThreadedRunnable runnable : runnables) {
- if (runnable.exception != null) {
- throw new RuntimeException(runnable.exception);
- }
- runnable.accessor.close();
- }
-
- assertEquals(10000, map.size());
-
-
-
- }
-
- @Test
- public void testNull() throws Exception {
-
- File path = new File(this.path, "testNull");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
- HashtableAccessor accessor = map.createAccessor(false);
-
- assertFalse(map.containsKey(accessor, 0));
- map.put(accessor, 0, "hello world");
- assertEquals("hello world", map.get(accessor, 0));
- String oldValue = map.put(accessor, 0, null);
- assertEquals("hello world", oldValue);
- assertNull(map.get(accessor, 0));
- assertEquals(1, map.size());
- assertTrue(map.containsKey(accessor, 0));
-
- accessor.close();
- }
@Test
public void testSimple() throws Exception {
- File path = new File(this.path, "testSimple");
+ File path = getDirectory("testSimple");
Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
+ HashtableAccessor accessor = map.createAccessor(false);
- assertEquals(0, map.size());
+ assertEquals(0, map.size(accessor));
- HashtableAccessor accessor = map.createAccessor(false);
assertNull(map.get(accessor, 0));
String v = "hello world";
assertNull(map.put(accessor, 0, v));
- assertEquals(1, map.size());
+ assertEquals(1, map.size(accessor));
assertEquals(v, map.get(accessor, 0));
assertNull(map.put(accessor, 1, v));
- assertEquals(2, map.size());
+ assertEquals(2, map.size(accessor));
String v2 = map.get(accessor, 1);
assertEquals(v, v2);
@@ -211,13 +79,14 @@
accessor.close();
- //
+ // close and reopen
map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
- assertEquals(2, map.size());
-
accessor = map.createAccessor(false);
+
+ assertEquals(2, map.size(accessor));
+
cursor = map.new Cursor(accessor);
assertTrue(cursor.next());
@@ -231,7 +100,7 @@
assertFalse(cursor.next());
v = map.remove(accessor, 0);
- assertEquals(1, map.size());
+ assertEquals(1, map.size(accessor));
assertEquals(v2, v);
assertNull(map.get(accessor, 0));
@@ -246,7 +115,7 @@
assertNull(map.get(accessor, 0));
assertNotNull(map.get(accessor, 1));
- assertEquals(1, map.size());
+ assertEquals(1, map.size(accessor));
for (int i = 2; i < 100; i++) {
map.put(accessor, i, v);
@@ -258,31 +127,22 @@
}
-
@Test
- public void testPartitions() throws Exception {
-
- // load lots of data to it in order to make sure it produce more than one partition but not too many.
-
- File path = new File(this.path, "testPartitions");
+ public void testNull() throws Exception {
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 10000, new IntegerHandler(), new StringValueHandler());
+ File path = getDirectory("testNull");
+ Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
HashtableAccessor accessor = map.createAccessor(false);
- StringBuilder sb = new StringBuilder(10000);
- for (int i = 0; i < 1000; i++) {
- sb.append("THIS WILL TURN OUT TO BE A RATHER LONG STRING");
- }
- String str = sb.toString();
- for (int i = 101; i < 5000; i++) {
- map.put(accessor, i, str + String.valueOf(i));
- }
- assertEquals(3, map.getPartitions().size());
-
- for (int i = 101; i < 5000; i++) {
- assertEquals(str + String.valueOf(i), map.get(accessor, i));
- }
+ assertFalse(map.containsKey(accessor, 0));
+ map.put(accessor, 0, "hello world");
+ assertEquals("hello world", map.get(accessor, 0));
+ String oldValue = map.put(accessor, 0, null);
+ assertEquals("hello world", oldValue);
+ assertNull(map.get(accessor, 0));
+ assertEquals(1, map.size(accessor));
+ assertTrue(map.containsKey(accessor, 0));
accessor.close();
}
Copied: labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestMultiThreaded.java (from r753612, labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java)
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestMultiThreaded.java?p2=labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestMultiThreaded.java&p1=labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java&r1=753612&r2=753638&rev=753638&view=diff
==============================================================================
--- labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java (original)
+++ labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestMultiThreaded.java Sat Mar 14 11:36:58 2009
@@ -35,54 +35,8 @@
* Date: 2009-jan-03
* Time: 05:43:56
*/
-public class TestHashtable extends TestCase {
+public class TestMultiThreaded extends HashtableTest {
- private File path;
-
- public TestHashtable() {
- path = new File("target/" + Hashtable.class.getSimpleName() + "/" + System.currentTimeMillis());
- if (!path.exists() && !path.mkdirs()) {
- throw new RuntimeException("Could not create path " + path);
- }
- }
-
-
-
-
- @Test
- public void testRehash() throws Exception {
-
- File path = new File(this.path, "testRehash");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
- HashtableAccessor accessor = map.createAccessor(false);
-
- map.put(accessor, 0, "hello world");
- map.put(accessor, 1, "greetings world");
- map.put(accessor, 5, "salutations world");
-
- accessor.close();
-
- map.rehash(1000);
-
- assertEquals(3, map.size());
-
- accessor = map.createAccessor(false);
- assertEquals("hello world", map.get(accessor, 0));
- assertEquals("greetings world", map.get(accessor, 1));
- assertEquals("salutations world", map.get(accessor, 5));
-
- assertEquals(3, map.size());
-
- // todo cursor
-
- accessor.close();
-
- // todo close and open again, then assert the same post-rehash tests once again
-
-
-
- }
private class TestThreadedRunnable implements Runnable {
@@ -101,8 +55,9 @@
public void run() {
try {
for (int i = 0; i < 1000; i++) {
- map.put(accessor, pk.getAndIncrement(), "foobar");
- System.out.println(id + "\t" + i);
+ int pk = this.pk.getAndIncrement();
+ map.put(accessor, pk, "foobar");
+ log.info(id + "\t" + i + "\t" + pk);
}
} catch (Exception exception) {
this.exception = exception;
@@ -113,7 +68,7 @@
@Test
public void testThreaded() throws Exception {
- File path = new File(this.path, "testThreaded");
+ File path = getDirectory("testThreaded");
Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 40000, new IntegerHandler(), new StringValueHandler());
map.setLockWaitTimeoutMilliseconds(60000);
@@ -144,148 +99,14 @@
runnable.accessor.close();
}
- assertEquals(10000, map.size());
-
-
-
- }
-
- @Test
- public void testNull() throws Exception {
-
- File path = new File(this.path, "testNull");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
HashtableAccessor accessor = map.createAccessor(false);
- assertFalse(map.containsKey(accessor, 0));
- map.put(accessor, 0, "hello world");
- assertEquals("hello world", map.get(accessor, 0));
- String oldValue = map.put(accessor, 0, null);
- assertEquals("hello world", oldValue);
- assertNull(map.get(accessor, 0));
- assertEquals(1, map.size());
- assertTrue(map.containsKey(accessor, 0));
+ assertEquals(10000, map.size(accessor));
accessor.close();
- }
-
- @Test
- public void testSimple() throws Exception {
-
- File path = new File(this.path, "testSimple");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
-
- assertEquals(0, map.size());
-
- HashtableAccessor accessor = map.createAccessor(false);
-
- assertNull(map.get(accessor, 0));
-
- String v = "hello world";
-
- assertNull(map.put(accessor, 0, v));
- assertEquals(1, map.size());
- assertEquals(v, map.get(accessor, 0));
- assertNull(map.put(accessor, 1, v));
- assertEquals(2, map.size());
- String v2 = map.get(accessor, 1);
- assertEquals(v, v2);
-
- // test cursor
- accessor = map.createAccessor(false);
- Hashtable<Integer, String>.Cursor cursor = map.new Cursor(accessor);
-
- assertTrue(cursor.next());
- assertEquals(0, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertTrue(cursor.next());
- assertEquals(1, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertFalse(cursor.next());
-
- accessor.close();
-
- //
-
- map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
-
- assertEquals(2, map.size());
-
- accessor = map.createAccessor(false);
- cursor = map.new Cursor(accessor);
-
- assertTrue(cursor.next());
- assertEquals(0, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertTrue(cursor.next());
- assertEquals(1, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertFalse(cursor.next());
-
- v = map.remove(accessor, 0);
- assertEquals(1, map.size());
- assertEquals(v2, v);
- assertNull(map.get(accessor, 0));
-
- accessor.close();
-
- //
-
- map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
-
- accessor = map.createAccessor(false);
-
- assertNull(map.get(accessor, 0));
-
- assertNotNull(map.get(accessor, 1));
- assertEquals(1, map.size());
-
- for (int i = 2; i < 100; i++) {
- map.put(accessor, i, v);
- assertEquals(v, map.get(accessor, i));
- }
-
- accessor.close();
-
-
- }
-
-
- @Test
- public void testPartitions() throws Exception {
-
- // load lots of data to it in order to make sure it produce more than one partition but not too many.
-
- File path = new File(this.path, "testPartitions");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 10000, new IntegerHandler(), new StringValueHandler());
-
- HashtableAccessor accessor = map.createAccessor(false);
-
- StringBuilder sb = new StringBuilder(10000);
- for (int i = 0; i < 1000; i++) {
- sb.append("THIS WILL TURN OUT TO BE A RATHER LONG STRING");
- }
- String str = sb.toString();
- for (int i = 101; i < 5000; i++) {
- map.put(accessor, i, str + String.valueOf(i));
- }
- assertEquals(3, map.getPartitions().size());
-
- for (int i = 101; i < 5000; i++) {
- assertEquals(str + String.valueOf(i), map.get(accessor, i));
- }
-
- accessor.close();
}
-}
+}
\ No newline at end of file
Copied: labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestRehash.java (from r753612, labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java)
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestRehash.java?p2=labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestRehash.java&p1=labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java&r1=753612&r2=753638&rev=753638&view=diff
==============================================================================
--- labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java (original)
+++ labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestRehash.java Sat Mar 14 11:36:58 2009
@@ -35,24 +35,13 @@
* Date: 2009-jan-03
* Time: 05:43:56
*/
-public class TestHashtable extends TestCase {
-
- private File path;
-
- public TestHashtable() {
- path = new File("target/" + Hashtable.class.getSimpleName() + "/" + System.currentTimeMillis());
- if (!path.exists() && !path.mkdirs()) {
- throw new RuntimeException("Could not create path " + path);
- }
- }
-
-
+public class TestRehash extends HashtableTest {
@Test
public void testRehash() throws Exception {
- File path = new File(this.path, "testRehash");
+ File path = getDirectory("testRehash");
Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
HashtableAccessor accessor = map.createAccessor(false);
@@ -61,231 +50,34 @@
map.put(accessor, 1, "greetings world");
map.put(accessor, 5, "salutations world");
- accessor.close();
+ assertEquals(3, map.size(accessor));
- map.rehash(1000);
-
- assertEquals(3, map.size());
-
- accessor = map.createAccessor(false);
assertEquals("hello world", map.get(accessor, 0));
assertEquals("greetings world", map.get(accessor, 1));
assertEquals("salutations world", map.get(accessor, 5));
- assertEquals(3, map.size());
-
- // todo cursor
-
- accessor.close();
-
- // todo close and open again, then assert the same post-rehash tests once again
-
-
-
- }
-
-
- private class TestThreadedRunnable implements Runnable {
- private Hashtable<Integer, String> map;
- private HashtableAccessor accessor;
- private Exception exception;
- private AtomicInteger pk;
- private int id;
-
- private TestThreadedRunnable(Hashtable<Integer, String> map, AtomicInteger pk) throws IOException {
- this.map = map;
- this.pk = pk;
- accessor = map.createAccessor(false);
- }
-
- public void run() {
- try {
- for (int i = 0; i < 1000; i++) {
- map.put(accessor, pk.getAndIncrement(), "foobar");
- System.out.println(id + "\t" + i);
- }
- } catch (Exception exception) {
- this.exception = exception;
- throw new RuntimeException(exception);
- }
- }
- }
-
- @Test
- public void testThreaded() throws Exception {
- File path = new File(this.path, "testThreaded");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 40000, new IntegerHandler(), new StringValueHandler());
- map.setLockWaitTimeoutMilliseconds(60000);
- AtomicInteger pk = new AtomicInteger(0);
-
- List<Thread> threads = new ArrayList<Thread>();
- List<TestThreadedRunnable> runnables = new ArrayList<TestThreadedRunnable>();
-
- for (int i = 0; i < 10; i++) {
- TestThreadedRunnable runnable = new TestThreadedRunnable(map, pk);
- runnable.id = i;
- runnables.add(runnable);
- Thread thread = new Thread(runnable);
- thread.setName("TestThread " + String.valueOf(i));
- threads.add(thread);
- }
- for (Thread thread : threads) {
- thread.start();
- }
- for (Thread thread : threads) {
- thread.join();
- }
-
- for (TestThreadedRunnable runnable : runnables) {
- if (runnable.exception != null) {
- throw new RuntimeException(runnable.exception);
- }
- runnable.accessor.close();
- }
-
- assertEquals(10000, map.size());
-
-
-
- }
-
- @Test
- public void testNull() throws Exception {
-
- File path = new File(this.path, "testNull");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
- HashtableAccessor accessor = map.createAccessor(false);
-
- assertFalse(map.containsKey(accessor, 0));
- map.put(accessor, 0, "hello world");
- assertEquals("hello world", map.get(accessor, 0));
- String oldValue = map.put(accessor, 0, null);
- assertEquals("hello world", oldValue);
- assertNull(map.get(accessor, 0));
- assertEquals(1, map.size());
- assertTrue(map.containsKey(accessor, 0));
-
- accessor.close();
- }
-
- @Test
- public void testSimple() throws Exception {
-
- File path = new File(this.path, "testSimple");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
-
- assertEquals(0, map.size());
-
- HashtableAccessor accessor = map.createAccessor(false);
-
- assertNull(map.get(accessor, 0));
-
- String v = "hello world";
-
- assertNull(map.put(accessor, 0, v));
- assertEquals(1, map.size());
- assertEquals(v, map.get(accessor, 0));
-
- assertNull(map.put(accessor, 1, v));
- assertEquals(2, map.size());
-
- String v2 = map.get(accessor, 1);
- assertEquals(v, v2);
- // test cursor
- accessor = map.createAccessor(false);
- Hashtable<Integer, String>.Cursor cursor = map.new Cursor(accessor);
-
- assertTrue(cursor.next());
- assertEquals(0, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertTrue(cursor.next());
- assertEquals(1, (int) cursor.key());
- assertEquals(v, cursor.value());
+ map.rehash(accessor, 1000);
- assertFalse(cursor.next());
+ assertEquals(3, map.size(accessor));
accessor.close();
- //
-
- map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
-
- assertEquals(2, map.size());
-
accessor = map.createAccessor(false);
- cursor = map.new Cursor(accessor);
-
- assertTrue(cursor.next());
- assertEquals(0, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertTrue(cursor.next());
- assertEquals(1, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertFalse(cursor.next());
-
- v = map.remove(accessor, 0);
- assertEquals(1, map.size());
- assertEquals(v2, v);
- assertNull(map.get(accessor, 0));
-
- accessor.close();
-
- //
-
- map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
-
- accessor = map.createAccessor(false);
-
- assertNull(map.get(accessor, 0));
+ assertEquals(3, map.size(accessor));
+ assertEquals("hello world", map.get(accessor, 0));
+ assertEquals("greetings world", map.get(accessor, 1));
+ assertEquals("salutations world", map.get(accessor, 5));
- assertNotNull(map.get(accessor, 1));
- assertEquals(1, map.size());
+ assertEquals(3, map.size(accessor));
- for (int i = 2; i < 100; i++) {
- map.put(accessor, i, v);
- assertEquals(v, map.get(accessor, i));
- }
+ // todo cursor
accessor.close();
+ // todo close and open again, then assert the same post-rehash tests once again
- }
-
-
- @Test
- public void testPartitions() throws Exception {
-
- // load lots of data to it in order to make sure it produce more than one partition but not too many.
-
- File path = new File(this.path, "testPartitions");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 10000, new IntegerHandler(), new StringValueHandler());
-
- HashtableAccessor accessor = map.createAccessor(false);
-
- StringBuilder sb = new StringBuilder(10000);
- for (int i = 0; i < 1000; i++) {
- sb.append("THIS WILL TURN OUT TO BE A RATHER LONG STRING");
- }
- String str = sb.toString();
- for (int i = 101; i < 5000; i++) {
- map.put(accessor, i, str + String.valueOf(i));
- }
- assertEquals(3, map.getPartitions().size());
-
- for (int i = 101; i < 5000; i++) {
- assertEquals(str + String.valueOf(i), map.get(accessor, i));
- }
- accessor.close();
}
-
-
-}
+
+}
\ No newline at end of file
Copied: labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestValuePartitions.java (from r753612, labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java)
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestValuePartitions.java?p2=labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestValuePartitions.java&p1=labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java&r1=753612&r2=753638&rev=753638&view=diff
==============================================================================
--- labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestHashtable.java (original)
+++ labs/bananadb/trunk/src/test/java/org/apache/labs/bananadb/hashtable/TestValuePartitions.java Sat Mar 14 11:36:58 2009
@@ -35,228 +35,7 @@
* Date: 2009-jan-03
* Time: 05:43:56
*/
-public class TestHashtable extends TestCase {
-
- private File path;
-
- public TestHashtable() {
- path = new File("target/" + Hashtable.class.getSimpleName() + "/" + System.currentTimeMillis());
- if (!path.exists() && !path.mkdirs()) {
- throw new RuntimeException("Could not create path " + path);
- }
- }
-
-
-
-
- @Test
- public void testRehash() throws Exception {
-
- File path = new File(this.path, "testRehash");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
- HashtableAccessor accessor = map.createAccessor(false);
-
- map.put(accessor, 0, "hello world");
- map.put(accessor, 1, "greetings world");
- map.put(accessor, 5, "salutations world");
-
- accessor.close();
-
- map.rehash(1000);
-
- assertEquals(3, map.size());
-
- accessor = map.createAccessor(false);
- assertEquals("hello world", map.get(accessor, 0));
- assertEquals("greetings world", map.get(accessor, 1));
- assertEquals("salutations world", map.get(accessor, 5));
-
- assertEquals(3, map.size());
-
- // todo cursor
-
- accessor.close();
-
- // todo close and open again, then assert the same post-rehash tests once again
-
-
-
- }
-
-
- private class TestThreadedRunnable implements Runnable {
- private Hashtable<Integer, String> map;
- private HashtableAccessor accessor;
- private Exception exception;
- private AtomicInteger pk;
- private int id;
-
- private TestThreadedRunnable(Hashtable<Integer, String> map, AtomicInteger pk) throws IOException {
- this.map = map;
- this.pk = pk;
- accessor = map.createAccessor(false);
- }
-
- public void run() {
- try {
- for (int i = 0; i < 1000; i++) {
- map.put(accessor, pk.getAndIncrement(), "foobar");
- System.out.println(id + "\t" + i);
- }
- } catch (Exception exception) {
- this.exception = exception;
- throw new RuntimeException(exception);
- }
- }
- }
-
- @Test
- public void testThreaded() throws Exception {
- File path = new File(this.path, "testThreaded");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 40000, new IntegerHandler(), new StringValueHandler());
- map.setLockWaitTimeoutMilliseconds(60000);
- AtomicInteger pk = new AtomicInteger(0);
-
- List<Thread> threads = new ArrayList<Thread>();
- List<TestThreadedRunnable> runnables = new ArrayList<TestThreadedRunnable>();
-
- for (int i = 0; i < 10; i++) {
- TestThreadedRunnable runnable = new TestThreadedRunnable(map, pk);
- runnable.id = i;
- runnables.add(runnable);
- Thread thread = new Thread(runnable);
- thread.setName("TestThread " + String.valueOf(i));
- threads.add(thread);
- }
- for (Thread thread : threads) {
- thread.start();
- }
- for (Thread thread : threads) {
- thread.join();
- }
-
- for (TestThreadedRunnable runnable : runnables) {
- if (runnable.exception != null) {
- throw new RuntimeException(runnable.exception);
- }
- runnable.accessor.close();
- }
-
- assertEquals(10000, map.size());
-
-
-
- }
-
- @Test
- public void testNull() throws Exception {
-
- File path = new File(this.path, "testNull");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
- HashtableAccessor accessor = map.createAccessor(false);
-
- assertFalse(map.containsKey(accessor, 0));
- map.put(accessor, 0, "hello world");
- assertEquals("hello world", map.get(accessor, 0));
- String oldValue = map.put(accessor, 0, null);
- assertEquals("hello world", oldValue);
- assertNull(map.get(accessor, 0));
- assertEquals(1, map.size());
- assertTrue(map.containsKey(accessor, 0));
-
- accessor.close();
- }
-
- @Test
- public void testSimple() throws Exception {
-
- File path = new File(this.path, "testSimple");
-
- Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
-
- assertEquals(0, map.size());
-
- HashtableAccessor accessor = map.createAccessor(false);
-
- assertNull(map.get(accessor, 0));
-
- String v = "hello world";
-
- assertNull(map.put(accessor, 0, v));
- assertEquals(1, map.size());
- assertEquals(v, map.get(accessor, 0));
-
- assertNull(map.put(accessor, 1, v));
- assertEquals(2, map.size());
-
- String v2 = map.get(accessor, 1);
- assertEquals(v, v2);
-
- // test cursor
- accessor = map.createAccessor(false);
- Hashtable<Integer, String>.Cursor cursor = map.new Cursor(accessor);
-
- assertTrue(cursor.next());
- assertEquals(0, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertTrue(cursor.next());
- assertEquals(1, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertFalse(cursor.next());
-
- accessor.close();
-
- //
-
- map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
-
- assertEquals(2, map.size());
-
- accessor = map.createAccessor(false);
- cursor = map.new Cursor(accessor);
-
- assertTrue(cursor.next());
- assertEquals(0, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertTrue(cursor.next());
- assertEquals(1, (int) cursor.key());
- assertEquals(v, cursor.value());
-
- assertFalse(cursor.next());
-
- v = map.remove(accessor, 0);
- assertEquals(1, map.size());
- assertEquals(v2, v);
- assertNull(map.get(accessor, 0));
-
- accessor.close();
-
- //
-
- map = new Hashtable<Integer, String>(path, 100, new IntegerHandler(), new StringValueHandler());
-
- accessor = map.createAccessor(false);
-
- assertNull(map.get(accessor, 0));
-
- assertNotNull(map.get(accessor, 1));
- assertEquals(1, map.size());
-
- for (int i = 2; i < 100; i++) {
- map.put(accessor, i, v);
- assertEquals(v, map.get(accessor, i));
- }
-
- accessor.close();
-
-
- }
+public class TestValuePartitions extends HashtableTest {
@Test
@@ -264,7 +43,7 @@
// load lots of data to it in order to make sure it produce more than one partition but not too many.
- File path = new File(this.path, "testPartitions");
+ File path = getDirectory("testPartitions");
Hashtable<Integer, String> map = new Hashtable<Integer, String>(path, 10000, new IntegerHandler(), new StringValueHandler());
@@ -288,4 +67,4 @@
}
-}
+}
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@labs.apache.org
For additional commands, e-mail: commits-help@labs.apache.org