svn commit: r532565 - in /lucene/solr/trunk/client/ruby/solr-ruby: lib/solr/importer/delimited_file_source.rb lib/solr/importer/tab_delimited_file_source.rb test/unit/delimited_file_source_test.rb test/unit/tab_delimited_file_source_test.rb

[eh...@apache.org]

Author: ehatcher
Date: Wed Apr 25 19:06:01 2007
New Revision: 532565

URL: http://svn.apache.org/viewvc?view=rev&rev=532565
Log:
rename tab_delimited to just delimited and added splitter parameter allowing lines to be split on any regular expression

Added:
    lucene/solr/trunk/client/ruby/solr-ruby/lib/solr/importer/delimited_file_source.rb   (with props)
    lucene/solr/trunk/client/ruby/solr-ruby/test/unit/delimited_file_source_test.rb   (with props)
Removed:
    lucene/solr/trunk/client/ruby/solr-ruby/lib/solr/importer/tab_delimited_file_source.rb
    lucene/solr/trunk/client/ruby/solr-ruby/test/unit/tab_delimited_file_source_test.rb

Added: lucene/solr/trunk/client/ruby/solr-ruby/lib/solr/importer/delimited_file_source.rb
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/client/ruby/solr-ruby/lib/solr/importer/delimited_file_source.rb?view=auto&rev=532565
==============================================================================
--- lucene/solr/trunk/client/ruby/solr-ruby/lib/solr/importer/delimited_file_source.rb (added)
+++ lucene/solr/trunk/client/ruby/solr-ruby/lib/solr/importer/delimited_file_source.rb Wed Apr 25 19:06:01 2007
@@ -0,0 +1,38 @@
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# For files with the first line containing field names
+# Currently not designed for enormous files, as all lines are
+# read into an array
+class Solr::Importer::DelimitedFileSource
+  include Enumerable
+  
+  def initialize(filename, splitter=/\t/)
+    @filename = filename
+    @splitter = splitter
+  end
+
+  def each
+    lines = IO.readlines(@filename)
+    headers = lines[0].split(@splitter).collect{|h| h.chomp}
+    
+    lines[1..-1].each do |line|
+      data = headers.zip(line.split(@splitter).collect{|s| s.chomp})
+      def data.[](key)
+        self.assoc(key.to_s)[1]
+      end
+      
+      yield(data)
+    end
+  end
+  
+end

Propchange: lucene/solr/trunk/client/ruby/solr-ruby/lib/solr/importer/delimited_file_source.rb
------------------------------------------------------------------------------
    svn:executable = *

Added: lucene/solr/trunk/client/ruby/solr-ruby/test/unit/delimited_file_source_test.rb
URL: http://svn.apache.org/viewvc/lucene/solr/trunk/client/ruby/solr-ruby/test/unit/delimited_file_source_test.rb?view=auto&rev=532565
==============================================================================
--- lucene/solr/trunk/client/ruby/solr-ruby/test/unit/delimited_file_source_test.rb (added)
+++ lucene/solr/trunk/client/ruby/solr-ruby/test/unit/delimited_file_source_test.rb Wed Apr 25 19:06:01 2007
@@ -0,0 +1,29 @@
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+require 'solr'
+require 'test/unit'
+
+class DelimitedFileSourceTest < Test::Unit::TestCase
+
+  def test_load
+    filename = File.expand_path(File.dirname(__FILE__)) + "/tab_delimited.txt"
+    
+    source = Solr::Importer::DelimitedFileSource.new(filename,/\t/)
+    assert_equal source.to_a.size, 1
+    
+    source.each do |data|
+       assert_equal data[:asin], '0865681740'
+    end
+  end
+
+end

Propchange: lucene/solr/trunk/client/ruby/solr-ruby/test/unit/delimited_file_source_test.rb
------------------------------------------------------------------------------
    svn:executable = *