You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Benedikt Ritter <br...@apache.org> on 2015/04/20 14:27:50 UTC
Re: svn commit: r1658276 - in /commons/proper/csv/trunk: pom.xml src/test/java/org/apache/commons/csv/CSVBenchmark.java
Hello again,
I've had some time to look ad the JMH Benchmarks. What I see is, that the
build now issues a warning:
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.commons:commons-csv:jar:1.1.1-SNAPSHOT
[WARNING] 'profiles.profile[benchmark].dependencies.dependency.systemPath'
for org.skife.kasparov:csv:jar should not point at files within the project
directory, ${basedir}/csv-1.0.jar will be unresolvable by dependent
projects @ line 432, column 23
[WARNING]
[WARNING] It is highly recommended to fix these problems because they
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
[WARNING]
Furthermore I can no longer run tests from within Intellij, because it can
not construct the class path for the JMH Benchmark class correctly. One
work around for this would be to create an additional source folder
"benchmark" and put the CSVBenchmark there.
WDYT?
Benedikt
2015-02-16 15:43 GMT+01:00 Ralph Goers <ra...@dslextreme.com>:
> JMH is a tool used during the build. The source code for the product does
> not depend on it. See http://www.apache.org/legal/resolved.html#prohibited
> <http://www.apache.org/legal/resolved.html#prohibited>.
>
> Ralph
>
>
> > On Feb 15, 2015, at 2:23 AM, Benedikt Ritter <br...@apache.org> wrote:
> >
> > 2015-02-09 2:40 GMT+01:00 sebb <se...@gmail.com>:
> >
> >> On 8 February 2015 at 23:47, <eb...@apache.org> wrote:
> >>> Author: ebourg
> >>> Date: Sun Feb 8 23:47:27 2015
> >>> New Revision: 1658276
> >>>
> >>> URL: http://svn.apache.org/r1658276
> >>> Log:
> >>> Added a JMH benchmark to compare the most commons CSV parsers
> >>
> >> -1
> >>
> >> This has broken the Continuum build.
> >>
> >> Also the JMH website says:
> >>
> >> "The recommended way to run a JMH benchmark is to use Maven to setup a
> >> standalone project that depends on the jar files of your application.
> >> This approach is preferred to ensure that the benchmarks are correctly
> >> initialized and produce reliable results. It is possible to run
> >> benchmarks from within an existing project, and even from within an
> >> IDE, however setup is more complex and the results are less reliable."
> >>
> >> Further, it's not clear to me what the JMH license is.
> >> It rather looks like GPL.
> >>
> >
> > +1 for this change in general. It's far superior to the manual benchmark
> we
> > had before.
> >
> > Regarding the problems mentioned by sebb:
> >
> > - Is the continuum build fixed already?
> > - if we add further functionality, we will eventually end up with a multi
> > module build that has modules like: core, beans, annotations, benchmarks.
> > For now we should follow KISS and simply add it to the project until
> > unresolvable problems show up.
> > - regarding the license issue: since log4j seems to use it, I think we
> are
> > on the save side. Maybe someone from the log4j team can comment (ping at
> > bodewig, ggregory)
> >
> >
> >>
> >>> Added:
> >>>
> >>
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> >> (with props)
> >>> Modified:
> >>> commons/proper/csv/trunk/pom.xml
> >>>
> >>> Modified: commons/proper/csv/trunk/pom.xml
> >>> URL:
> >>
> http://svn.apache.org/viewvc/commons/proper/csv/trunk/pom.xml?rev=1658276&r1=1658275&r2=1658276&view=diff
> >>>
> >>
> ==============================================================================
> >>> --- commons/proper/csv/trunk/pom.xml (original)
> >>> +++ commons/proper/csv/trunk/pom.xml Sun Feb 8 23:47:27 2015
> >>> @@ -73,6 +73,12 @@ CSV files of various types.
> >>> <organization>The Apache Software Foundation</organization>
> >>> </developer>
> >>> <developer>
> >>> + <name>Emmanuel Bourg</name>
> >>> + <id>ebourg</id>
> >>> + <email>ebourg@apache.org</email>
> >>> + <organization>Apache</organization>
> >>> + </developer>
> >>> + <developer>
> >>> <name>Gary Gregory</name>
> >>> <id>ggregory</id>
> >>> <email>ggregory@apache.org</email>
> >>> @@ -357,6 +363,112 @@ CSV files of various types.
> >>> </configuration>
> >>> </execution>
> >>> </executions>
> >>> + </plugin>
> >>> + </plugins>
> >>> + </build>
> >>> + </profile>
> >>> +
> >>> +
> >>> + <!-- Profile to build and run the benchmarks. Use 'mvn test
> >> -Pbenchmark', and add '-Dbenchmark=foo' to run only the foo benchmark
> -->
> >>> + <profile>
> >>> + <id>benchmark</id>
> >>> +
> >>> + <dependencies>
> >>> + <dependency>
> >>> + <groupId>org.openjdk.jmh</groupId>
> >>> + <artifactId>jmh-core</artifactId>
> >>> + <version>1.5.2</version>
> >>> + <scope>test</scope>
> >>> + </dependency>
> >>> +
> >>> + <dependency>
> >>> + <groupId>org.openjdk.jmh</groupId>
> >>> + <artifactId>jmh-generator-annprocess</artifactId>
> >>> + <version>1.5.2</version>
> >>> + <scope>test</scope>
> >>> + </dependency>
> >>> +
> >>> + <dependency>
> >>> + <groupId>genjava</groupId>
> >>> + <artifactId>gj-csv</artifactId>
> >>> + <version>1.0</version>
> >>> + <scope>test</scope>
> >>> + </dependency>
> >>> +
> >>> + <dependency>
> >>> + <groupId>net.sourceforge.javacsv</groupId>
> >>> + <artifactId>javacsv</artifactId>
> >>> + <version>2.0</version>
> >>> + <scope>test</scope>
> >>> + </dependency>
> >>> +
> >>> + <dependency>
> >>> + <groupId>com.opencsv</groupId>
> >>> + <artifactId>opencsv</artifactId>
> >>> + <version>3.1</version>
> >>> + <scope>test</scope>
> >>> + </dependency>
> >>> +
> >>> + <dependency>
> >>> + <groupId>net.sf.supercsv</groupId>
> >>> + <artifactId>super-csv</artifactId>
> >>> + <version>2.2.1</version>
> >>> + </dependency>
> >>> +
> >>> + <!-- Not in Maven Central, download manually from
> >> http://kasparov.skife.org/csv/csv-1.0.jar and copy in the base
> directory
> >> -->
> >>> + <dependency>
> >>> + <groupId>org.skife.kasparov</groupId>
> >>> + <artifactId>csv</artifactId>
> >>> + <version>1.0</version>
> >>> + <scope>system</scope>
> >>> + <systemPath>${basedir}/csv-1.0.jar</systemPath>
> >>> + </dependency>
> >>> + </dependencies>
> >>> +
> >>> + <properties>
> >>> + <skipTests>true</skipTests>
> >>> + <benchmark>org.apache</benchmark>
> >>> + </properties>
> >>> +
> >>> + <build>
> >>> + <plugins>
> >>> + <!-- Enable the compilation of the benchmarks -->
> >>> + <plugin>
> >>> + <artifactId>maven-compiler-plugin</artifactId>
> >>> + <configuration combine.self="override">
> >>> + <testIncludes>
> >>> + <testInclude>**/*</testInclude>
> >>> + </testIncludes>
> >>> + </configuration>
> >>> + </plugin>
> >>> +
> >>> + <!-- Hook the benchmarks to the test phase -->
> >>> + <plugin>
> >>> + <groupId>org.codehaus.mojo</groupId>
> >>> + <artifactId>exec-maven-plugin</artifactId>
> >>> + <executions>
> >>> + <execution>
> >>> + <id>benchmark</id>
> >>> + <phase>test</phase>
> >>> + <goals>
> >>> + <goal>exec</goal>
> >>> + </goals>
> >>> + <configuration>
> >>> + <classpathScope>test</classpathScope>
> >>> + <executable>java</executable>
> >>> + <arguments>
> >>> + <argument>-classpath</argument>
> >>> + <classpath/>
> >>> + <argument>org.openjdk.jmh.Main</argument>
> >>> + <argument>-rf</argument>
> >>> + <argument>json</argument>
> >>> + <argument>-rff</argument>
> >>> + <argument>target/jmh-result.json</argument>
> >>> + <argument>${benchmark}</argument>
> >>> + </arguments>
> >>> + </configuration>
> >>> + </execution>
> >>> + </executions>
> >>> </plugin>
> >>> </plugins>
> >>> </build>
> >>>
> >>> Added:
> >>
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> >>> URL:
> >>
> http://svn.apache.org/viewvc/commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java?rev=1658276&view=auto
> >>>
> >>
> ==============================================================================
> >>> ---
> >>
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> >> (added)
> >>> +++
> >>
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> >> Sun Feb 8 23:47:27 2015
> >>> @@ -0,0 +1,172 @@
> >>> +/*
> >>> + * Licensed to the Apache Software Foundation (ASF) under one or more
> >>> + * contributor license agreements. See the NOTICE file distributed
> with
> >>> + * this work for additional information regarding copyright ownership.
> >>> + * The ASF licenses this file to You under the Apache License, Version
> >> 2.0
> >>> + * (the "License"); you may not use this file except in compliance
> with
> >>> + * the License. You may obtain a copy of the License at
> >>> + *
> >>> + * http://www.apache.org/licenses/LICENSE-2.0
> >>> + *
> >>> + * Unless required by applicable law or agreed to in writing, software
> >>> + * distributed under the License is distributed on an "AS IS" BASIS,
> >>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> >> implied.
> >>> + * See the License for the specific language governing permissions and
> >>> + * limitations under the License.
> >>> + */
> >>> +
> >>> +package org.apache.commons.csv;
> >>> +
> >>> +import java.io.BufferedReader;
> >>> +import java.io.FileReader;
> >>> +import java.io.IOException;
> >>> +import java.util.List;
> >>> +import java.util.concurrent.TimeUnit;
> >>> +
> >>> +import com.generationjava.io.CsvReader;
> >>> +import org.openjdk.jmh.annotations.Benchmark;
> >>> +import org.openjdk.jmh.annotations.BenchmarkMode;
> >>> +import org.openjdk.jmh.annotations.Fork;
> >>> +import org.openjdk.jmh.annotations.Measurement;
> >>> +import org.openjdk.jmh.annotations.Mode;
> >>> +import org.openjdk.jmh.annotations.OutputTimeUnit;
> >>> +import org.openjdk.jmh.annotations.Threads;
> >>> +import org.openjdk.jmh.annotations.Warmup;
> >>> +import org.openjdk.jmh.infra.Blackhole;
> >>> +import org.supercsv.io.CsvListReader;
> >>> +import org.supercsv.prefs.CsvPreference;
> >>> +
> >>> +@BenchmarkMode(Mode.AverageTime)
> >>> +@Fork(value = 1, jvmArgs = "-server")
> >>> +@Threads(1)
> >>> +@Warmup(iterations = 10)
> >>> +@Measurement(iterations = 10)
> >>> +@OutputTimeUnit(TimeUnit.MILLISECONDS)
> >>> +public class CSVBenchmark {
> >>> +
> >>> + private BufferedReader getReader() throws IOException {
> >>> + return new BufferedReader(new
> FileReader("worldcitiespop.txt"));
> >>> + }
> >>> +
> >>> + @Benchmark
> >>> + public int baseline(Blackhole bh) throws Exception {
> >>> + BufferedReader in = getReader();
> >>> + int count = 0;
> >>> + String line;
> >>> + while ((line = in.readLine()) != null) {
> >>> + count++;
> >>> + }
> >>> +
> >>> + bh.consume(count);
> >>> + in.close();
> >>> + return count;
> >>> + }
> >>> +
> >>> + @Benchmark
> >>> + public int parseCommonsCSV(Blackhole bh) throws Exception {
> >>> + BufferedReader in = getReader();
> >>> +
> >>> + CSVFormat format = CSVFormat.DEFAULT.withHeader();
> >>> +
> >>> + int count = 0;
> >>> + for (CSVRecord record : format.parse(in)) {
> >>> + count++;
> >>> + }
> >>> +
> >>> + bh.consume(count);
> >>> + in.close();
> >>> + return count;
> >>> + }
> >>> +
> >>> + @Benchmark
> >>> + public int parseGenJavaCSV(Blackhole bh) throws Exception {
> >>> + BufferedReader in = getReader();
> >>> +
> >>> + CsvReader reader = new CsvReader(in);
> >>> + reader.setFieldDelimiter(',');
> >>> +
> >>> + int count = 0;
> >>> + String[] record = null;
> >>> + while ((record = reader.readLine()) != null) {
> >>> + count++;
> >>> + }
> >>> +
> >>> + bh.consume(count);
> >>> + in.close();
> >>> + return count;
> >>> + }
> >>> +
> >>> + @Benchmark
> >>> + public int parseJavaCSV(Blackhole bh) throws Exception {
> >>> + BufferedReader in = getReader();
> >>> +
> >>> + com.csvreader.CsvReader reader = new
> >> com.csvreader.CsvReader(in, ',');
> >>> + reader.setRecordDelimiter('\n');
> >>> +
> >>> + int count = 0;
> >>> + while (reader.readRecord()) {
> >>> + count++;
> >>> + }
> >>> +
> >>> + bh.consume(count);
> >>> + in.close();
> >>> + return count;
> >>> + }
> >>> +
> >>> + @Benchmark
> >>> + public int parseOpenCSV(Blackhole bh) throws Exception {
> >>> + BufferedReader in = getReader();
> >>> +
> >>> + com.opencsv.CSVReader reader = new com.opencsv.CSVReader(in,
> >> ',');
> >>> +
> >>> + int count = 0;
> >>> + while (reader.readNext() != null) {
> >>> + count++;
> >>> + }
> >>> +
> >>> + bh.consume(count);
> >>> + in.close();
> >>> + return count;
> >>> + }
> >>> +
> >>> + @Benchmark
> >>> + public int parseSkifeCSV(Blackhole bh) throws Exception {
> >>> + BufferedReader in = getReader();
> >>> +
> >>> + org.skife.csv.CSVReader reader = new
> >> org.skife.csv.SimpleReader();
> >>> + reader.setSeperator(',');
> >>> +
> >>> + CountingReaderCallback callback = new
> CountingReaderCallback();
> >>> + reader.parse(in, callback);
> >>> +
> >>> + bh.consume(callback);
> >>> + in.close();
> >>> + return callback.count;
> >>> + }
> >>> +
> >>> + private static class CountingReaderCallback implements
> >> org.skife.csv.ReaderCallback {
> >>> + public int count = 0;
> >>> +
> >>> + @Override
> >>> + public void onRow(String[] fields) {
> >>> + count++;
> >>> + }
> >>> + }
> >>> +
> >>> + @Benchmark
> >>> + public int parseSuperCSV(Blackhole bh) throws Exception {
> >>> + BufferedReader in = getReader();
> >>> +
> >>> + CsvListReader reader = new CsvListReader(in,
> >> CsvPreference.STANDARD_PREFERENCE);
> >>> +
> >>> + int count = 0;
> >>> + List<String> record = null;
> >>> + while ((record = reader.read()) != null) {
> >>> + count++;
> >>> + }
> >>> +
> >>> + bh.consume(count);
> >>> + in.close();
> >>> + return count;
> >>> + }
> >>> +}
> >>>
> >>> Propchange:
> >>
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> >>>
> >>
> ------------------------------------------------------------------------------
> >>> svn:eol-style = native
> >>>
> >>> Propchange:
> >>
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> >>>
> >>
> ------------------------------------------------------------------------------
> >>> svn:keywords = Date Author Id Revision HeadURL
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >>
> >
> >
> > --
> > http://people.apache.org/~britter/
> > http://www.systemoutprintln.de/
> > http://twitter.com/BenediktRitter
> > http://github.com/britter
>
>
--
http://people.apache.org/~britter/
http://www.systemoutprintln.de/
http://twitter.com/BenediktRitter
http://github.com/britter
Re: svn commit: r1658276 - in /commons/proper/csv/trunk: pom.xml src/test/java/org/apache/commons/csv/CSVBenchmark.java
Posted by Benedikt Ritter <br...@apache.org>.
Hello,
2015-04-20 17:08 GMT+02:00 Gary Gregory <ga...@gmail.com>:
> Hm... In Log4j, we have a separate Maven module for benchmarks, but we
> already have a lot of modules in Log4j... I'm sure we do not want another
> module for [cvs].
>
I've started work on a Csv2BeanMapper, which could be a separate module...
>
> We have a source folder for tests already, a third one might require a lot
> more configuration in the POM, I'm not sure. Isn't there a way to solve
> this with the current project structure?
>
There are more issues IMHO. One has to manually download one of the
benchmark dependencies, because it is not available at maven central. I was
able to get my IDE to compile again, but we should integrate the download
into the build. And we should resolve the warnings which show up at the
start of the build.
B.
>
> Gary
>
> On Mon, Apr 20, 2015 at 5:27 AM, Benedikt Ritter <br...@apache.org>
> wrote:
>
> > Hello again,
> >
> > I've had some time to look ad the JMH Benchmarks. What I see is, that the
> > build now issues a warning:
> >
> > [WARNING]
> > [WARNING] Some problems were encountered while building the effective
> model
> > for org.apache.commons:commons-csv:jar:1.1.1-SNAPSHOT
> > [WARNING]
> 'profiles.profile[benchmark].dependencies.dependency.systemPath'
> > for org.skife.kasparov:csv:jar should not point at files within the
> project
> > directory, ${basedir}/csv-1.0.jar will be unresolvable by dependent
> > projects @ line 432, column 23
> > [WARNING]
> > [WARNING] It is highly recommended to fix these problems because they
> > threaten the stability of your build.
> > [WARNING]
> > [WARNING] For this reason, future Maven versions might no longer support
> > building such malformed projects.
> > [WARNING]
> >
> > Furthermore I can no longer run tests from within Intellij, because it
> can
> > not construct the class path for the JMH Benchmark class correctly. One
> > work around for this would be to create an additional source folder
> > "benchmark" and put the CSVBenchmark there.
> >
> > WDYT?
> >
> > Benedikt
> >
> > 2015-02-16 15:43 GMT+01:00 Ralph Goers <ra...@dslextreme.com>:
> >
> > > JMH is a tool used during the build. The source code for the product
> does
> > > not depend on it. See
> > http://www.apache.org/legal/resolved.html#prohibited
> > > <http://www.apache.org/legal/resolved.html#prohibited>.
> > >
> > > Ralph
> > >
> > >
> > > > On Feb 15, 2015, at 2:23 AM, Benedikt Ritter <br...@apache.org>
> > wrote:
> > > >
> > > > 2015-02-09 2:40 GMT+01:00 sebb <se...@gmail.com>:
> > > >
> > > >> On 8 February 2015 at 23:47, <eb...@apache.org> wrote:
> > > >>> Author: ebourg
> > > >>> Date: Sun Feb 8 23:47:27 2015
> > > >>> New Revision: 1658276
> > > >>>
> > > >>> URL: http://svn.apache.org/r1658276
> > > >>> Log:
> > > >>> Added a JMH benchmark to compare the most commons CSV parsers
> > > >>
> > > >> -1
> > > >>
> > > >> This has broken the Continuum build.
> > > >>
> > > >> Also the JMH website says:
> > > >>
> > > >> "The recommended way to run a JMH benchmark is to use Maven to
> setup a
> > > >> standalone project that depends on the jar files of your
> application.
> > > >> This approach is preferred to ensure that the benchmarks are
> correctly
> > > >> initialized and produce reliable results. It is possible to run
> > > >> benchmarks from within an existing project, and even from within an
> > > >> IDE, however setup is more complex and the results are less
> reliable."
> > > >>
> > > >> Further, it's not clear to me what the JMH license is.
> > > >> It rather looks like GPL.
> > > >>
> > > >
> > > > +1 for this change in general. It's far superior to the manual
> > benchmark
> > > we
> > > > had before.
> > > >
> > > > Regarding the problems mentioned by sebb:
> > > >
> > > > - Is the continuum build fixed already?
> > > > - if we add further functionality, we will eventually end up with a
> > multi
> > > > module build that has modules like: core, beans, annotations,
> > benchmarks.
> > > > For now we should follow KISS and simply add it to the project until
> > > > unresolvable problems show up.
> > > > - regarding the license issue: since log4j seems to use it, I think
> we
> > > are
> > > > on the save side. Maybe someone from the log4j team can comment (ping
> > at
> > > > bodewig, ggregory)
> > > >
> > > >
> > > >>
> > > >>> Added:
> > > >>>
> > > >>
> > >
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > > >> (with props)
> > > >>> Modified:
> > > >>> commons/proper/csv/trunk/pom.xml
> > > >>>
> > > >>> Modified: commons/proper/csv/trunk/pom.xml
> > > >>> URL:
> > > >>
> > >
> >
> http://svn.apache.org/viewvc/commons/proper/csv/trunk/pom.xml?rev=1658276&r1=1658275&r2=1658276&view=diff
> > > >>>
> > > >>
> > >
> >
> ==============================================================================
> > > >>> --- commons/proper/csv/trunk/pom.xml (original)
> > > >>> +++ commons/proper/csv/trunk/pom.xml Sun Feb 8 23:47:27 2015
> > > >>> @@ -73,6 +73,12 @@ CSV files of various types.
> > > >>> <organization>The Apache Software Foundation</organization>
> > > >>> </developer>
> > > >>> <developer>
> > > >>> + <name>Emmanuel Bourg</name>
> > > >>> + <id>ebourg</id>
> > > >>> + <email>ebourg@apache.org</email>
> > > >>> + <organization>Apache</organization>
> > > >>> + </developer>
> > > >>> + <developer>
> > > >>> <name>Gary Gregory</name>
> > > >>> <id>ggregory</id>
> > > >>> <email>ggregory@apache.org</email>
> > > >>> @@ -357,6 +363,112 @@ CSV files of various types.
> > > >>> </configuration>
> > > >>> </execution>
> > > >>> </executions>
> > > >>> + </plugin>
> > > >>> + </plugins>
> > > >>> + </build>
> > > >>> + </profile>
> > > >>> +
> > > >>> +
> > > >>> + <!-- Profile to build and run the benchmarks. Use 'mvn test
> > > >> -Pbenchmark', and add '-Dbenchmark=foo' to run only the foo
> benchmark
> > > -->
> > > >>> + <profile>
> > > >>> + <id>benchmark</id>
> > > >>> +
> > > >>> + <dependencies>
> > > >>> + <dependency>
> > > >>> + <groupId>org.openjdk.jmh</groupId>
> > > >>> + <artifactId>jmh-core</artifactId>
> > > >>> + <version>1.5.2</version>
> > > >>> + <scope>test</scope>
> > > >>> + </dependency>
> > > >>> +
> > > >>> + <dependency>
> > > >>> + <groupId>org.openjdk.jmh</groupId>
> > > >>> + <artifactId>jmh-generator-annprocess</artifactId>
> > > >>> + <version>1.5.2</version>
> > > >>> + <scope>test</scope>
> > > >>> + </dependency>
> > > >>> +
> > > >>> + <dependency>
> > > >>> + <groupId>genjava</groupId>
> > > >>> + <artifactId>gj-csv</artifactId>
> > > >>> + <version>1.0</version>
> > > >>> + <scope>test</scope>
> > > >>> + </dependency>
> > > >>> +
> > > >>> + <dependency>
> > > >>> + <groupId>net.sourceforge.javacsv</groupId>
> > > >>> + <artifactId>javacsv</artifactId>
> > > >>> + <version>2.0</version>
> > > >>> + <scope>test</scope>
> > > >>> + </dependency>
> > > >>> +
> > > >>> + <dependency>
> > > >>> + <groupId>com.opencsv</groupId>
> > > >>> + <artifactId>opencsv</artifactId>
> > > >>> + <version>3.1</version>
> > > >>> + <scope>test</scope>
> > > >>> + </dependency>
> > > >>> +
> > > >>> + <dependency>
> > > >>> + <groupId>net.sf.supercsv</groupId>
> > > >>> + <artifactId>super-csv</artifactId>
> > > >>> + <version>2.2.1</version>
> > > >>> + </dependency>
> > > >>> +
> > > >>> + <!-- Not in Maven Central, download manually from
> > > >> http://kasparov.skife.org/csv/csv-1.0.jar and copy in the base
> > > directory
> > > >> -->
> > > >>> + <dependency>
> > > >>> + <groupId>org.skife.kasparov</groupId>
> > > >>> + <artifactId>csv</artifactId>
> > > >>> + <version>1.0</version>
> > > >>> + <scope>system</scope>
> > > >>> + <systemPath>${basedir}/csv-1.0.jar</systemPath>
> > > >>> + </dependency>
> > > >>> + </dependencies>
> > > >>> +
> > > >>> + <properties>
> > > >>> + <skipTests>true</skipTests>
> > > >>> + <benchmark>org.apache</benchmark>
> > > >>> + </properties>
> > > >>> +
> > > >>> + <build>
> > > >>> + <plugins>
> > > >>> + <!-- Enable the compilation of the benchmarks -->
> > > >>> + <plugin>
> > > >>> + <artifactId>maven-compiler-plugin</artifactId>
> > > >>> + <configuration combine.self="override">
> > > >>> + <testIncludes>
> > > >>> + <testInclude>**/*</testInclude>
> > > >>> + </testIncludes>
> > > >>> + </configuration>
> > > >>> + </plugin>
> > > >>> +
> > > >>> + <!-- Hook the benchmarks to the test phase -->
> > > >>> + <plugin>
> > > >>> + <groupId>org.codehaus.mojo</groupId>
> > > >>> + <artifactId>exec-maven-plugin</artifactId>
> > > >>> + <executions>
> > > >>> + <execution>
> > > >>> + <id>benchmark</id>
> > > >>> + <phase>test</phase>
> > > >>> + <goals>
> > > >>> + <goal>exec</goal>
> > > >>> + </goals>
> > > >>> + <configuration>
> > > >>> + <classpathScope>test</classpathScope>
> > > >>> + <executable>java</executable>
> > > >>> + <arguments>
> > > >>> + <argument>-classpath</argument>
> > > >>> + <classpath/>
> > > >>> + <argument>org.openjdk.jmh.Main</argument>
> > > >>> + <argument>-rf</argument>
> > > >>> + <argument>json</argument>
> > > >>> + <argument>-rff</argument>
> > > >>> + <argument>target/jmh-result.json</argument>
> > > >>> + <argument>${benchmark}</argument>
> > > >>> + </arguments>
> > > >>> + </configuration>
> > > >>> + </execution>
> > > >>> + </executions>
> > > >>> </plugin>
> > > >>> </plugins>
> > > >>> </build>
> > > >>>
> > > >>> Added:
> > > >>
> > >
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > > >>> URL:
> > > >>
> > >
> >
> http://svn.apache.org/viewvc/commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java?rev=1658276&view=auto
> > > >>>
> > > >>
> > >
> >
> ==============================================================================
> > > >>> ---
> > > >>
> > >
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > > >> (added)
> > > >>> +++
> > > >>
> > >
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > > >> Sun Feb 8 23:47:27 2015
> > > >>> @@ -0,0 +1,172 @@
> > > >>> +/*
> > > >>> + * Licensed to the Apache Software Foundation (ASF) under one or
> > more
> > > >>> + * contributor license agreements. See the NOTICE file
> distributed
> > > with
> > > >>> + * this work for additional information regarding copyright
> > ownership.
> > > >>> + * The ASF licenses this file to You under the Apache License,
> > Version
> > > >> 2.0
> > > >>> + * (the "License"); you may not use this file except in compliance
> > > with
> > > >>> + * the License. You may obtain a copy of the License at
> > > >>> + *
> > > >>> + * http://www.apache.org/licenses/LICENSE-2.0
> > > >>> + *
> > > >>> + * Unless required by applicable law or agreed to in writing,
> > software
> > > >>> + * distributed under the License is distributed on an "AS IS"
> BASIS,
> > > >>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > > >> implied.
> > > >>> + * See the License for the specific language governing permissions
> > and
> > > >>> + * limitations under the License.
> > > >>> + */
> > > >>> +
> > > >>> +package org.apache.commons.csv;
> > > >>> +
> > > >>> +import java.io.BufferedReader;
> > > >>> +import java.io.FileReader;
> > > >>> +import java.io.IOException;
> > > >>> +import java.util.List;
> > > >>> +import java.util.concurrent.TimeUnit;
> > > >>> +
> > > >>> +import com.generationjava.io.CsvReader;
> > > >>> +import org.openjdk.jmh.annotations.Benchmark;
> > > >>> +import org.openjdk.jmh.annotations.BenchmarkMode;
> > > >>> +import org.openjdk.jmh.annotations.Fork;
> > > >>> +import org.openjdk.jmh.annotations.Measurement;
> > > >>> +import org.openjdk.jmh.annotations.Mode;
> > > >>> +import org.openjdk.jmh.annotations.OutputTimeUnit;
> > > >>> +import org.openjdk.jmh.annotations.Threads;
> > > >>> +import org.openjdk.jmh.annotations.Warmup;
> > > >>> +import org.openjdk.jmh.infra.Blackhole;
> > > >>> +import org.supercsv.io.CsvListReader;
> > > >>> +import org.supercsv.prefs.CsvPreference;
> > > >>> +
> > > >>> +@BenchmarkMode(Mode.AverageTime)
> > > >>> +@Fork(value = 1, jvmArgs = "-server")
> > > >>> +@Threads(1)
> > > >>> +@Warmup(iterations = 10)
> > > >>> +@Measurement(iterations = 10)
> > > >>> +@OutputTimeUnit(TimeUnit.MILLISECONDS)
> > > >>> +public class CSVBenchmark {
> > > >>> +
> > > >>> + private BufferedReader getReader() throws IOException {
> > > >>> + return new BufferedReader(new
> > > FileReader("worldcitiespop.txt"));
> > > >>> + }
> > > >>> +
> > > >>> + @Benchmark
> > > >>> + public int baseline(Blackhole bh) throws Exception {
> > > >>> + BufferedReader in = getReader();
> > > >>> + int count = 0;
> > > >>> + String line;
> > > >>> + while ((line = in.readLine()) != null) {
> > > >>> + count++;
> > > >>> + }
> > > >>> +
> > > >>> + bh.consume(count);
> > > >>> + in.close();
> > > >>> + return count;
> > > >>> + }
> > > >>> +
> > > >>> + @Benchmark
> > > >>> + public int parseCommonsCSV(Blackhole bh) throws Exception {
> > > >>> + BufferedReader in = getReader();
> > > >>> +
> > > >>> + CSVFormat format = CSVFormat.DEFAULT.withHeader();
> > > >>> +
> > > >>> + int count = 0;
> > > >>> + for (CSVRecord record : format.parse(in)) {
> > > >>> + count++;
> > > >>> + }
> > > >>> +
> > > >>> + bh.consume(count);
> > > >>> + in.close();
> > > >>> + return count;
> > > >>> + }
> > > >>> +
> > > >>> + @Benchmark
> > > >>> + public int parseGenJavaCSV(Blackhole bh) throws Exception {
> > > >>> + BufferedReader in = getReader();
> > > >>> +
> > > >>> + CsvReader reader = new CsvReader(in);
> > > >>> + reader.setFieldDelimiter(',');
> > > >>> +
> > > >>> + int count = 0;
> > > >>> + String[] record = null;
> > > >>> + while ((record = reader.readLine()) != null) {
> > > >>> + count++;
> > > >>> + }
> > > >>> +
> > > >>> + bh.consume(count);
> > > >>> + in.close();
> > > >>> + return count;
> > > >>> + }
> > > >>> +
> > > >>> + @Benchmark
> > > >>> + public int parseJavaCSV(Blackhole bh) throws Exception {
> > > >>> + BufferedReader in = getReader();
> > > >>> +
> > > >>> + com.csvreader.CsvReader reader = new
> > > >> com.csvreader.CsvReader(in, ',');
> > > >>> + reader.setRecordDelimiter('\n');
> > > >>> +
> > > >>> + int count = 0;
> > > >>> + while (reader.readRecord()) {
> > > >>> + count++;
> > > >>> + }
> > > >>> +
> > > >>> + bh.consume(count);
> > > >>> + in.close();
> > > >>> + return count;
> > > >>> + }
> > > >>> +
> > > >>> + @Benchmark
> > > >>> + public int parseOpenCSV(Blackhole bh) throws Exception {
> > > >>> + BufferedReader in = getReader();
> > > >>> +
> > > >>> + com.opencsv.CSVReader reader = new
> com.opencsv.CSVReader(in,
> > > >> ',');
> > > >>> +
> > > >>> + int count = 0;
> > > >>> + while (reader.readNext() != null) {
> > > >>> + count++;
> > > >>> + }
> > > >>> +
> > > >>> + bh.consume(count);
> > > >>> + in.close();
> > > >>> + return count;
> > > >>> + }
> > > >>> +
> > > >>> + @Benchmark
> > > >>> + public int parseSkifeCSV(Blackhole bh) throws Exception {
> > > >>> + BufferedReader in = getReader();
> > > >>> +
> > > >>> + org.skife.csv.CSVReader reader = new
> > > >> org.skife.csv.SimpleReader();
> > > >>> + reader.setSeperator(',');
> > > >>> +
> > > >>> + CountingReaderCallback callback = new
> > > CountingReaderCallback();
> > > >>> + reader.parse(in, callback);
> > > >>> +
> > > >>> + bh.consume(callback);
> > > >>> + in.close();
> > > >>> + return callback.count;
> > > >>> + }
> > > >>> +
> > > >>> + private static class CountingReaderCallback implements
> > > >> org.skife.csv.ReaderCallback {
> > > >>> + public int count = 0;
> > > >>> +
> > > >>> + @Override
> > > >>> + public void onRow(String[] fields) {
> > > >>> + count++;
> > > >>> + }
> > > >>> + }
> > > >>> +
> > > >>> + @Benchmark
> > > >>> + public int parseSuperCSV(Blackhole bh) throws Exception {
> > > >>> + BufferedReader in = getReader();
> > > >>> +
> > > >>> + CsvListReader reader = new CsvListReader(in,
> > > >> CsvPreference.STANDARD_PREFERENCE);
> > > >>> +
> > > >>> + int count = 0;
> > > >>> + List<String> record = null;
> > > >>> + while ((record = reader.read()) != null) {
> > > >>> + count++;
> > > >>> + }
> > > >>> +
> > > >>> + bh.consume(count);
> > > >>> + in.close();
> > > >>> + return count;
> > > >>> + }
> > > >>> +}
> > > >>>
> > > >>> Propchange:
> > > >>
> > >
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > > >>>
> > > >>
> > >
> >
> ------------------------------------------------------------------------------
> > > >>> svn:eol-style = native
> > > >>>
> > > >>> Propchange:
> > > >>
> > >
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > > >>>
> > > >>
> > >
> >
> ------------------------------------------------------------------------------
> > > >>> svn:keywords = Date Author Id Revision HeadURL
> > > >>>
> > > >>>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > >> For additional commands, e-mail: dev-help@commons.apache.org
> > > >>
> > > >>
> > > >
> > > >
> > > > --
> > > > http://people.apache.org/~britter/
> > > > http://www.systemoutprintln.de/
> > > > http://twitter.com/BenediktRitter
> > > > http://github.com/britter
> > >
> > >
> >
> >
> > --
> > http://people.apache.org/~britter/
> > http://www.systemoutprintln.de/
> > http://twitter.com/BenediktRitter
> > http://github.com/britter
> >
>
>
>
> --
> E-Mail: garydgregory@gmail.com | ggregory@apache.org
> Java Persistence with Hibernate, Second Edition
> <http://www.manning.com/bauer3/>
> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
> Spring Batch in Action <http://www.manning.com/templier/>
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>
--
http://people.apache.org/~britter/
http://www.systemoutprintln.de/
http://twitter.com/BenediktRitter
http://github.com/britter
Re: svn commit: r1658276 - in /commons/proper/csv/trunk: pom.xml src/test/java/org/apache/commons/csv/CSVBenchmark.java
Posted by Gary Gregory <ga...@gmail.com>.
Hm... In Log4j, we have a separate Maven module for benchmarks, but we
already have a lot of modules in Log4j... I'm sure we do not want another
module for [cvs].
We have a source folder for tests already, a third one might require a lot
more configuration in the POM, I'm not sure. Isn't there a way to solve
this with the current project structure?
Gary
On Mon, Apr 20, 2015 at 5:27 AM, Benedikt Ritter <br...@apache.org> wrote:
> Hello again,
>
> I've had some time to look ad the JMH Benchmarks. What I see is, that the
> build now issues a warning:
>
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.commons:commons-csv:jar:1.1.1-SNAPSHOT
> [WARNING] 'profiles.profile[benchmark].dependencies.dependency.systemPath'
> for org.skife.kasparov:csv:jar should not point at files within the project
> directory, ${basedir}/csv-1.0.jar will be unresolvable by dependent
> projects @ line 432, column 23
> [WARNING]
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
> [WARNING]
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
> [WARNING]
>
> Furthermore I can no longer run tests from within Intellij, because it can
> not construct the class path for the JMH Benchmark class correctly. One
> work around for this would be to create an additional source folder
> "benchmark" and put the CSVBenchmark there.
>
> WDYT?
>
> Benedikt
>
> 2015-02-16 15:43 GMT+01:00 Ralph Goers <ra...@dslextreme.com>:
>
> > JMH is a tool used during the build. The source code for the product does
> > not depend on it. See
> http://www.apache.org/legal/resolved.html#prohibited
> > <http://www.apache.org/legal/resolved.html#prohibited>.
> >
> > Ralph
> >
> >
> > > On Feb 15, 2015, at 2:23 AM, Benedikt Ritter <br...@apache.org>
> wrote:
> > >
> > > 2015-02-09 2:40 GMT+01:00 sebb <se...@gmail.com>:
> > >
> > >> On 8 February 2015 at 23:47, <eb...@apache.org> wrote:
> > >>> Author: ebourg
> > >>> Date: Sun Feb 8 23:47:27 2015
> > >>> New Revision: 1658276
> > >>>
> > >>> URL: http://svn.apache.org/r1658276
> > >>> Log:
> > >>> Added a JMH benchmark to compare the most commons CSV parsers
> > >>
> > >> -1
> > >>
> > >> This has broken the Continuum build.
> > >>
> > >> Also the JMH website says:
> > >>
> > >> "The recommended way to run a JMH benchmark is to use Maven to setup a
> > >> standalone project that depends on the jar files of your application.
> > >> This approach is preferred to ensure that the benchmarks are correctly
> > >> initialized and produce reliable results. It is possible to run
> > >> benchmarks from within an existing project, and even from within an
> > >> IDE, however setup is more complex and the results are less reliable."
> > >>
> > >> Further, it's not clear to me what the JMH license is.
> > >> It rather looks like GPL.
> > >>
> > >
> > > +1 for this change in general. It's far superior to the manual
> benchmark
> > we
> > > had before.
> > >
> > > Regarding the problems mentioned by sebb:
> > >
> > > - Is the continuum build fixed already?
> > > - if we add further functionality, we will eventually end up with a
> multi
> > > module build that has modules like: core, beans, annotations,
> benchmarks.
> > > For now we should follow KISS and simply add it to the project until
> > > unresolvable problems show up.
> > > - regarding the license issue: since log4j seems to use it, I think we
> > are
> > > on the save side. Maybe someone from the log4j team can comment (ping
> at
> > > bodewig, ggregory)
> > >
> > >
> > >>
> > >>> Added:
> > >>>
> > >>
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > >> (with props)
> > >>> Modified:
> > >>> commons/proper/csv/trunk/pom.xml
> > >>>
> > >>> Modified: commons/proper/csv/trunk/pom.xml
> > >>> URL:
> > >>
> >
> http://svn.apache.org/viewvc/commons/proper/csv/trunk/pom.xml?rev=1658276&r1=1658275&r2=1658276&view=diff
> > >>>
> > >>
> >
> ==============================================================================
> > >>> --- commons/proper/csv/trunk/pom.xml (original)
> > >>> +++ commons/proper/csv/trunk/pom.xml Sun Feb 8 23:47:27 2015
> > >>> @@ -73,6 +73,12 @@ CSV files of various types.
> > >>> <organization>The Apache Software Foundation</organization>
> > >>> </developer>
> > >>> <developer>
> > >>> + <name>Emmanuel Bourg</name>
> > >>> + <id>ebourg</id>
> > >>> + <email>ebourg@apache.org</email>
> > >>> + <organization>Apache</organization>
> > >>> + </developer>
> > >>> + <developer>
> > >>> <name>Gary Gregory</name>
> > >>> <id>ggregory</id>
> > >>> <email>ggregory@apache.org</email>
> > >>> @@ -357,6 +363,112 @@ CSV files of various types.
> > >>> </configuration>
> > >>> </execution>
> > >>> </executions>
> > >>> + </plugin>
> > >>> + </plugins>
> > >>> + </build>
> > >>> + </profile>
> > >>> +
> > >>> +
> > >>> + <!-- Profile to build and run the benchmarks. Use 'mvn test
> > >> -Pbenchmark', and add '-Dbenchmark=foo' to run only the foo benchmark
> > -->
> > >>> + <profile>
> > >>> + <id>benchmark</id>
> > >>> +
> > >>> + <dependencies>
> > >>> + <dependency>
> > >>> + <groupId>org.openjdk.jmh</groupId>
> > >>> + <artifactId>jmh-core</artifactId>
> > >>> + <version>1.5.2</version>
> > >>> + <scope>test</scope>
> > >>> + </dependency>
> > >>> +
> > >>> + <dependency>
> > >>> + <groupId>org.openjdk.jmh</groupId>
> > >>> + <artifactId>jmh-generator-annprocess</artifactId>
> > >>> + <version>1.5.2</version>
> > >>> + <scope>test</scope>
> > >>> + </dependency>
> > >>> +
> > >>> + <dependency>
> > >>> + <groupId>genjava</groupId>
> > >>> + <artifactId>gj-csv</artifactId>
> > >>> + <version>1.0</version>
> > >>> + <scope>test</scope>
> > >>> + </dependency>
> > >>> +
> > >>> + <dependency>
> > >>> + <groupId>net.sourceforge.javacsv</groupId>
> > >>> + <artifactId>javacsv</artifactId>
> > >>> + <version>2.0</version>
> > >>> + <scope>test</scope>
> > >>> + </dependency>
> > >>> +
> > >>> + <dependency>
> > >>> + <groupId>com.opencsv</groupId>
> > >>> + <artifactId>opencsv</artifactId>
> > >>> + <version>3.1</version>
> > >>> + <scope>test</scope>
> > >>> + </dependency>
> > >>> +
> > >>> + <dependency>
> > >>> + <groupId>net.sf.supercsv</groupId>
> > >>> + <artifactId>super-csv</artifactId>
> > >>> + <version>2.2.1</version>
> > >>> + </dependency>
> > >>> +
> > >>> + <!-- Not in Maven Central, download manually from
> > >> http://kasparov.skife.org/csv/csv-1.0.jar and copy in the base
> > directory
> > >> -->
> > >>> + <dependency>
> > >>> + <groupId>org.skife.kasparov</groupId>
> > >>> + <artifactId>csv</artifactId>
> > >>> + <version>1.0</version>
> > >>> + <scope>system</scope>
> > >>> + <systemPath>${basedir}/csv-1.0.jar</systemPath>
> > >>> + </dependency>
> > >>> + </dependencies>
> > >>> +
> > >>> + <properties>
> > >>> + <skipTests>true</skipTests>
> > >>> + <benchmark>org.apache</benchmark>
> > >>> + </properties>
> > >>> +
> > >>> + <build>
> > >>> + <plugins>
> > >>> + <!-- Enable the compilation of the benchmarks -->
> > >>> + <plugin>
> > >>> + <artifactId>maven-compiler-plugin</artifactId>
> > >>> + <configuration combine.self="override">
> > >>> + <testIncludes>
> > >>> + <testInclude>**/*</testInclude>
> > >>> + </testIncludes>
> > >>> + </configuration>
> > >>> + </plugin>
> > >>> +
> > >>> + <!-- Hook the benchmarks to the test phase -->
> > >>> + <plugin>
> > >>> + <groupId>org.codehaus.mojo</groupId>
> > >>> + <artifactId>exec-maven-plugin</artifactId>
> > >>> + <executions>
> > >>> + <execution>
> > >>> + <id>benchmark</id>
> > >>> + <phase>test</phase>
> > >>> + <goals>
> > >>> + <goal>exec</goal>
> > >>> + </goals>
> > >>> + <configuration>
> > >>> + <classpathScope>test</classpathScope>
> > >>> + <executable>java</executable>
> > >>> + <arguments>
> > >>> + <argument>-classpath</argument>
> > >>> + <classpath/>
> > >>> + <argument>org.openjdk.jmh.Main</argument>
> > >>> + <argument>-rf</argument>
> > >>> + <argument>json</argument>
> > >>> + <argument>-rff</argument>
> > >>> + <argument>target/jmh-result.json</argument>
> > >>> + <argument>${benchmark}</argument>
> > >>> + </arguments>
> > >>> + </configuration>
> > >>> + </execution>
> > >>> + </executions>
> > >>> </plugin>
> > >>> </plugins>
> > >>> </build>
> > >>>
> > >>> Added:
> > >>
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > >>> URL:
> > >>
> >
> http://svn.apache.org/viewvc/commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java?rev=1658276&view=auto
> > >>>
> > >>
> >
> ==============================================================================
> > >>> ---
> > >>
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > >> (added)
> > >>> +++
> > >>
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > >> Sun Feb 8 23:47:27 2015
> > >>> @@ -0,0 +1,172 @@
> > >>> +/*
> > >>> + * Licensed to the Apache Software Foundation (ASF) under one or
> more
> > >>> + * contributor license agreements. See the NOTICE file distributed
> > with
> > >>> + * this work for additional information regarding copyright
> ownership.
> > >>> + * The ASF licenses this file to You under the Apache License,
> Version
> > >> 2.0
> > >>> + * (the "License"); you may not use this file except in compliance
> > with
> > >>> + * the License. You may obtain a copy of the License at
> > >>> + *
> > >>> + * http://www.apache.org/licenses/LICENSE-2.0
> > >>> + *
> > >>> + * Unless required by applicable law or agreed to in writing,
> software
> > >>> + * distributed under the License is distributed on an "AS IS" BASIS,
> > >>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > >> implied.
> > >>> + * See the License for the specific language governing permissions
> and
> > >>> + * limitations under the License.
> > >>> + */
> > >>> +
> > >>> +package org.apache.commons.csv;
> > >>> +
> > >>> +import java.io.BufferedReader;
> > >>> +import java.io.FileReader;
> > >>> +import java.io.IOException;
> > >>> +import java.util.List;
> > >>> +import java.util.concurrent.TimeUnit;
> > >>> +
> > >>> +import com.generationjava.io.CsvReader;
> > >>> +import org.openjdk.jmh.annotations.Benchmark;
> > >>> +import org.openjdk.jmh.annotations.BenchmarkMode;
> > >>> +import org.openjdk.jmh.annotations.Fork;
> > >>> +import org.openjdk.jmh.annotations.Measurement;
> > >>> +import org.openjdk.jmh.annotations.Mode;
> > >>> +import org.openjdk.jmh.annotations.OutputTimeUnit;
> > >>> +import org.openjdk.jmh.annotations.Threads;
> > >>> +import org.openjdk.jmh.annotations.Warmup;
> > >>> +import org.openjdk.jmh.infra.Blackhole;
> > >>> +import org.supercsv.io.CsvListReader;
> > >>> +import org.supercsv.prefs.CsvPreference;
> > >>> +
> > >>> +@BenchmarkMode(Mode.AverageTime)
> > >>> +@Fork(value = 1, jvmArgs = "-server")
> > >>> +@Threads(1)
> > >>> +@Warmup(iterations = 10)
> > >>> +@Measurement(iterations = 10)
> > >>> +@OutputTimeUnit(TimeUnit.MILLISECONDS)
> > >>> +public class CSVBenchmark {
> > >>> +
> > >>> + private BufferedReader getReader() throws IOException {
> > >>> + return new BufferedReader(new
> > FileReader("worldcitiespop.txt"));
> > >>> + }
> > >>> +
> > >>> + @Benchmark
> > >>> + public int baseline(Blackhole bh) throws Exception {
> > >>> + BufferedReader in = getReader();
> > >>> + int count = 0;
> > >>> + String line;
> > >>> + while ((line = in.readLine()) != null) {
> > >>> + count++;
> > >>> + }
> > >>> +
> > >>> + bh.consume(count);
> > >>> + in.close();
> > >>> + return count;
> > >>> + }
> > >>> +
> > >>> + @Benchmark
> > >>> + public int parseCommonsCSV(Blackhole bh) throws Exception {
> > >>> + BufferedReader in = getReader();
> > >>> +
> > >>> + CSVFormat format = CSVFormat.DEFAULT.withHeader();
> > >>> +
> > >>> + int count = 0;
> > >>> + for (CSVRecord record : format.parse(in)) {
> > >>> + count++;
> > >>> + }
> > >>> +
> > >>> + bh.consume(count);
> > >>> + in.close();
> > >>> + return count;
> > >>> + }
> > >>> +
> > >>> + @Benchmark
> > >>> + public int parseGenJavaCSV(Blackhole bh) throws Exception {
> > >>> + BufferedReader in = getReader();
> > >>> +
> > >>> + CsvReader reader = new CsvReader(in);
> > >>> + reader.setFieldDelimiter(',');
> > >>> +
> > >>> + int count = 0;
> > >>> + String[] record = null;
> > >>> + while ((record = reader.readLine()) != null) {
> > >>> + count++;
> > >>> + }
> > >>> +
> > >>> + bh.consume(count);
> > >>> + in.close();
> > >>> + return count;
> > >>> + }
> > >>> +
> > >>> + @Benchmark
> > >>> + public int parseJavaCSV(Blackhole bh) throws Exception {
> > >>> + BufferedReader in = getReader();
> > >>> +
> > >>> + com.csvreader.CsvReader reader = new
> > >> com.csvreader.CsvReader(in, ',');
> > >>> + reader.setRecordDelimiter('\n');
> > >>> +
> > >>> + int count = 0;
> > >>> + while (reader.readRecord()) {
> > >>> + count++;
> > >>> + }
> > >>> +
> > >>> + bh.consume(count);
> > >>> + in.close();
> > >>> + return count;
> > >>> + }
> > >>> +
> > >>> + @Benchmark
> > >>> + public int parseOpenCSV(Blackhole bh) throws Exception {
> > >>> + BufferedReader in = getReader();
> > >>> +
> > >>> + com.opencsv.CSVReader reader = new com.opencsv.CSVReader(in,
> > >> ',');
> > >>> +
> > >>> + int count = 0;
> > >>> + while (reader.readNext() != null) {
> > >>> + count++;
> > >>> + }
> > >>> +
> > >>> + bh.consume(count);
> > >>> + in.close();
> > >>> + return count;
> > >>> + }
> > >>> +
> > >>> + @Benchmark
> > >>> + public int parseSkifeCSV(Blackhole bh) throws Exception {
> > >>> + BufferedReader in = getReader();
> > >>> +
> > >>> + org.skife.csv.CSVReader reader = new
> > >> org.skife.csv.SimpleReader();
> > >>> + reader.setSeperator(',');
> > >>> +
> > >>> + CountingReaderCallback callback = new
> > CountingReaderCallback();
> > >>> + reader.parse(in, callback);
> > >>> +
> > >>> + bh.consume(callback);
> > >>> + in.close();
> > >>> + return callback.count;
> > >>> + }
> > >>> +
> > >>> + private static class CountingReaderCallback implements
> > >> org.skife.csv.ReaderCallback {
> > >>> + public int count = 0;
> > >>> +
> > >>> + @Override
> > >>> + public void onRow(String[] fields) {
> > >>> + count++;
> > >>> + }
> > >>> + }
> > >>> +
> > >>> + @Benchmark
> > >>> + public int parseSuperCSV(Blackhole bh) throws Exception {
> > >>> + BufferedReader in = getReader();
> > >>> +
> > >>> + CsvListReader reader = new CsvListReader(in,
> > >> CsvPreference.STANDARD_PREFERENCE);
> > >>> +
> > >>> + int count = 0;
> > >>> + List<String> record = null;
> > >>> + while ((record = reader.read()) != null) {
> > >>> + count++;
> > >>> + }
> > >>> +
> > >>> + bh.consume(count);
> > >>> + in.close();
> > >>> + return count;
> > >>> + }
> > >>> +}
> > >>>
> > >>> Propchange:
> > >>
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > >>>
> > >>
> >
> ------------------------------------------------------------------------------
> > >>> svn:eol-style = native
> > >>>
> > >>> Propchange:
> > >>
> >
> commons/proper/csv/trunk/src/test/java/org/apache/commons/csv/CSVBenchmark.java
> > >>>
> > >>
> >
> ------------------------------------------------------------------------------
> > >>> svn:keywords = Date Author Id Revision HeadURL
> > >>>
> > >>>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >> For additional commands, e-mail: dev-help@commons.apache.org
> > >>
> > >>
> > >
> > >
> > > --
> > > http://people.apache.org/~britter/
> > > http://www.systemoutprintln.de/
> > > http://twitter.com/BenediktRitter
> > > http://github.com/britter
> >
> >
>
>
> --
> http://people.apache.org/~britter/
> http://www.systemoutprintln.de/
> http://twitter.com/BenediktRitter
> http://github.com/britter
>
--
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory