GitLab will be upgraded to the 12.10.14-ce.0 on 28 Sept 2020 at 2.00pm (AEDT) to 2.30pm (AEDT). During the update, GitLab and Mattermost services will not be available. If you have any concerns with this, please talk to us at N110 (b) CSIT building.

Commit 2baec49b authored by Steve Blackburn's avatar Steve Blackburn Committed by GitHub

Merge pull request #125 from johnjiabinzhang/release-uncalibrated-9.13

Version bump for all 9.12 benchmarks, validated on OpenJDK 8
parents 509d5539 6139b5a9
before_script:
- export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
- java -version
- ant -version
- cd benchmarks
- touch local.properties
- echo "make=/usr/bin/make" >> local.properties
- echo "build.failonerror=true" >> local.properties
- echo "jdk7home=/usr/lib/jvm/java-7-openjdk-amd64" >> local.properties
- echo "jdk9home=/usr/lib/jvm/java-9-openjdk-amd64" >> local.properties
- cat local.properties
avrora:
script:
- ant -Dbuild.target-jar=dacapo.jar avrora
- java -jar dacapo.jar avrora
batik:
script:
- ant -Dbuild.target-jar=dacapo.jar batik
- java -jar dacapo.jar batik
eclipse:
script:
- ant -Dbuild.target-jar=dacapo.jar eclipse
- java -jar dacapo.jar eclipse
fop:
script:
- ant -Dbuild.target-jar=dacapo.jar fop
- java -jar dacapo.jar fop
h2:
script:
- ant -Dbuild.target-jar=dacapo.jar h2
- java -jar dacapo.jar h2
jython:
script:
- ant -Dbuild.target-jar=dacapo.jar jython
- java -jar dacapo.jar jython
luindex:
script:
- ant -Dbuild.target-jar=dacapo.jar luindex
- java -jar dacapo.jar luindex
lusearch:
script:
- ant -Dbuild.target-jar=dacapo.jar lusearch
- java -jar dacapo.jar lusearch
pmd:
script:
- ant -Dbuild.target-jar=dacapo.jar pmd
- java -jar dacapo.jar pmd
sunflow:
script:
- ant -Dbuild.target-jar=dacapo.jar sunflow
- java -jar dacapo.jar sunflow
tomcat:
script:
- ant -Dbuild.target-jar=dacapo.jar tomcat
- java -jar dacapo.jar tomcat
tradebeans:
script:
- script -q -c "ant -Dbuild.target-jar=dacapo.jar tradebeans" script.out
- sleep 120 | script -q -c "java -jar dacapo.jar tradebeans" script.out
tradesoap:
script:
- script -q -c "ant -Dbuild.target-jar=dacapo.jar tradesoap" script.out
- sleep 120 | script -q -c "java -jar dacapo.jar tradesoap" script.out
xalan:
script:
- ant -Dbuild.target-jar=dacapo.jar xalan
- java -jar dacapo.jar xalan
# The DaCapo Benchmark Suite
Last updated 2018-01-13
Last updated 2018-04-06
This benchmark suite is intend as a tool for the research community.
It consists of a set of open source, real world applications with
......@@ -29,9 +29,9 @@ The easiest way to obtain the benchmark suite is to download the pre-built jar f
If, however, you want to build from source read on...
The suite is built using ant 1.9 (1.10 and later will fail for many of the benchmarks). You will need the following tools:
The suite is built using ant 1.10. You will need the following tools:
* *[ant 1.9](http://ant.apache.org)* You need to install this yourself if you don't already have it.
* *[ant 1.10](http://ant.apache.org)* You need to install this yourself if you don't already have it.
* *[javacc](http://javacc.dev.java.net/)* Included in our tools directory.
......@@ -41,8 +41,23 @@ The suite is built using ant 1.9 (1.10 and later will fail for many of the bench
* *[svn](http://subversion.apache.org)*
* *[hg](https://www.mercurial-scm.org)*
** Run `ant -p` in the benchmarks directory for more information on building **.
**NOTE**
1.JDK 8 is needed to build the whole suite.
2.Building DayTrader and PMD using JDK 8 also requires JDK 7 to be installed, and jdk7home properly set. (see 2 below)
**IMPORTANT:** before trying to build the suite:
1. Set your `JAVA_HOME` environment variable appropriately (it must be set and be consistent with the VM that will be used to build the suite).
2. Copy `default.properties` to `local.properties` and edit it for your environment.
* Specifically, you must set `jdk7home` to point to a Java 7 installation.
For more information, run `ant -p` in the benchmarks directory.
......
- Update benchmark versions
avrora - from 20091224 to 20131011
batik - from 1.7 to 1.9
eclipse - from 3.5.1 to 4.7.2
fop - from 0.95 to 2.2
h2 - from 1.2.123 to 1.4.196
jython - from 2.5.1 to 2.7.1
luindex - from 2.4 to 7.1
lusearch - from 2.4 to 7.1
pmd - from 4.2.5 to 5.7.0*
tomcat - from 6 to 9
daytrader - from 2.1.7 to 3.0.0
xalan - from 2.7.1 to 2.7.2
- Update benchmark versions
antlr - from 2.4.2 to 2.4.7
......@@ -36,4 +49,4 @@ Release 2006-10-MR2
- Added "-ignoreValidation" command line switch.
- Changed jikesrvm stubs so that MMTk callback works with recent
JikesRVM releases (and hopefuly now with future JVMs that use MMTk).
\ No newline at end of file
JikesRVM releases (and hopefuly now with future JVMs that use MMTk).
......@@ -72,8 +72,25 @@ current state of deployed Java applications.
3. Changes
----------
3.0. Changes introduced by 9.12-MR1 (since 9.12)
3.0. Changes introduced by 9.13
Benchmark updates:
- avrora: 20091224 -> 20131011
- batik: 1.7 -> 1.9
- eclipse: 3.5.1 -> 4.7.2
- fop: 0.95 -> 2.2
- h2: 1.5 -> 1.8
- jython: 2.5.2 -> 2.7.1
- pmd: 4.2.5 -> 6.1.0
- tomcat: 6.0.2 -> 9.0.2
- daytrader: 2.4.1 -> 3.0-SNAPSHOT
- Apache Geronimo: 2.4.1 -> 3.0.1
- xalan: 2.7.1 -> 2.7.2
- lucene: 2.4.1 -> 7.1
Shipped tooling update:
- Maven: 2.1.0 -> 3.5.2
3.1. Changes introduced by 9.12-MR1 (since 9.12)
lusearch-fix is introduced as a new benchmark. The lusearch-fix
and lusearch benchmarks differ by a single line of code. This
......@@ -91,7 +108,7 @@ current state of deployed Java applications.
the suite builds with Java 8 VMs.
3.1. Benchmark additions since 2006-10-MR2
3.2. Benchmark additions since 2006-10-MR2
avrora: AVRORA is a set of simulation and analysis tools in a
framework for AVR micro-controllers. The benchmark
......@@ -134,7 +151,7 @@ current state of deployed Java applications.
two systems.
3.2. Benchmark deletions
3.3. Benchmark deletions
antlr: Antlr is single threaded and highly repetitive. The
most recent version of jython uses antlr; so antlr
......@@ -162,13 +179,13 @@ current state of deployed Java applications.
performing database engine.
3.3. Benchmark updates
3.4. Benchmark updates
All other benchmarks have been updated to reflect the latest release
of the underlying application.
3.4. Other Notable Changes
3.5. Other Notable Changes
The packaging of the DaCapo suite has been completely re-worked and
the source code is entirely re-organized.
......
......@@ -2,39 +2,34 @@ RELEASE PROCEDURES
0. Update and commit the release notes.
1. Determine a version number and optionally a nickname
export DACAPO_VERSION=9.12
export DACAPO_NICKNAME=bach
1. Determine a version number and optionally a nickname, specify that in dacapo.properties file
dacapo.version=3.27
dacapo.nickname=uncalibrated
2. Make fresh svn checkout.
mkdir svn
cd svn
svn co https://dacapobench.svn.sourceforge.net/svnroot/dacapobench/benchmarks/trunk dacapobench
2. Make fresh git clone
git clone https://github.com/dacapobench/dacapobench.git
3. Produced a source build and rename it:
cd dacapobench/benchmarks
ant -Dbuild.version=$DACAPO_VERSION -Dbuild.nickname=$DACAPO_NICKNAME source
mv dacapo-src.zip ../../../dacapo-$DACAPO_VERSION-$DACAPO_NICKNAME-src.zip
cd ../../..
ant -Dbuild.is-official=true source
mv dacapo-${dacapo.version}-${dacapo.nickname}-src.zip ..
cd ..
4. Create a new directory and unzip the new source
mkdir release
cd release
unzip ../dacapo-$DACAPO_VERSION-$DACAPO_NICKNAME-src.zip
unzip ../dacapo-${dacapo.version}-${dacapo.nickname}-src.zip
5. Build the new source (using a proxy if necessary)
export JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home
export ANT_OPTS="-Xmx512M -Dhttp.proxyHost=150.203.163.152 -Dhttp.proxyPort=3128"
export MAVEN_OPTS="-Xmx512M"
cd benchmarks
ant -Dbuild.version=$DACAPO_VERSION -Dbuild.nickname=$DACAPO_NICKNAME
ant -Dbuild.is-official=true
6. Rename the jar
mv dacapo-<xxx>.jar ../../dacapo-$DACAPO_VERSION-$DACAPO_NICKNAME.jar
6. Test the resulting jar
7. Test the resulting jar
8. Upload files
7. Upload files
- Log into the sourceforge site as a developer
- Go to Develop tab
- Go to Project Admin -> File Manager tab
......@@ -45,6 +40,6 @@ or
scp <filename> <sf-username>,dacapobench@frs.sourceforge.net:/home/frs/project/d/da/dacapobench/<release dir>
- set file properties via the file manager to identify release notes (left-click file names to set properties)
9. Ensure default download is correct (use left click on file to make it default for a given platform / platforms)
8. Ensure default download is correct (use left click on file to make it default for a given platform / platforms)
- https://sourceforge.net/projects/dacapobench/ (see default download here)
- https://sourceforge.net/projects/dacapobench/files/
benchmark avrora
class org.dacapo.harness.Avrora
thread-model single
jars "avrora-cvs-20091224.jar";
jars "avrora-cvs-20131011.jar";
size small args "-seconds=30",
"-platform=mica2",
......@@ -51,6 +51,6 @@ description
author "Ben L. Titzer",
license "UCLA",
url "http://compilers.cs.ucla.edu/avrora/",
version "cvs-20090612",
version "cvs-20131011",
threads "Externally single-threaded. Internally multi-threaded. Threads represent each simulated entity, so much fine-grained parallelism internally.",
repeats "Each execution simulates a range of microcontroller functions.";
......@@ -11,9 +11,12 @@
<property file="dacapo.properties" />
<property name="bm-name" value="avrora" />
<!--property name="bm-version" value="1.7.117" />
<property name="bm-url" value="https://nchc.dl.sourceforge.net/project/${bm-name}" />
<property name="bm-src" value="avrora-beta-${bm-version}.jar" />-->
<property name="bm-cvs-root" value=":pserver:anonymous@avrora.cvs.sourceforge.net:/cvsroot/avrora" />
<property name="bm-cvs-date" value="20091224" />
<property name="bm-cvs-date" value="20131011" />
<property name="bm-cvs-package" value="${bm-name}" />
<property name="bm-version" value="cvs-${bm-cvs-date}" />
<property name="bm-src" value="${bm-name}-${bm-version}-src.tar.gz" />
......
2b56066e8a0671d057a7c792dba79d4b
\ No newline at end of file
fb52a894f54c08277a13444126ab8a7b
\ No newline at end of file
......@@ -15,8 +15,8 @@ import org.dacapo.harness.Benchmark;
import org.dacapo.parser.Config;
/**
* @date $Date: 2009-12-24 11:19:36 +1100 (Thu, 24 Dec 2009) $
* @id $Id: Avrora.java 738 2009-12-24 00:19:36Z steveb-oss $
* date: $Date: 2009-12-24 11:19:36 +1100 (Thu, 24 Dec 2009) $
* id: $Id: Avrora.java 738 2009-12-24 00:19:36Z steveb-oss $
*/
public class Avrora extends Benchmark {
......
benchmark batik
class org.dacapo.harness.Batik
thread-model single
jars "batik-all.jar", "xml-apis-ext.jar","xml-apis.jar", "crimson-1.1.3.jar", "xerces_2_5_0.jar", "xalan-2.6.0.jar";
jars "batik-all-1.9.jar", "xml-apis-ext-1.3.04.jar","xml-apis-1.3.04.jar", "crimson-1.1.3.jar", "xercesImpl-2.9.1.jar", "xalan-2.7.2.jar", "serializer-2.7.2.jar", "xmlgraphics-commons-2.2.jar";
/*
* Validation currently limited to checking existence of output file
......@@ -40,6 +40,6 @@ description
copyright "Copyright (C) 2000 The Apache Software Foundation",
license "Apache License, Version 2.0",
url "http://xml.apache.org/batik/",
version "1.7",
version "1.9",
threads "Externally single-threaded. Internally multi-threaded.";
--- batik-1.9/batik-dom/src/main/java/org/apache/batik/dom/util/SAXDocumentFactory.java 2017-03-17 15:55:00.000000000 +1100
+++ ../SAXDocumentFactory.java 2017-12-01 12:13:20.811378286 +1100
@@ -399,7 +399,7 @@
static SAXParserFactory saxFactory;
static {
saxFactory = SAXParserFactory.newInstance();
- try {
+ /*try {
saxFactory.setFeature("http://xml.org/sax/features/external-general-entities", false);
saxFactory.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
saxFactory.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
@@ -409,7 +409,7 @@
e.printStackTrace();
} catch (ParserConfigurationException e) {
e.printStackTrace();
- }
+ }*/
}
/**
@@ -445,9 +445,9 @@
true);
parser.setFeature("http://xml.org/sax/features/validation",
isValidating);
- parser.setFeature("http://xml.org/sax/features/external-general-entities", false);
+ /*parser.setFeature("http://xml.org/sax/features/external-general-entities", false);
parser.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
- parser.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
+ parser.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);*/
parser.setProperty("http://xml.org/sax/properties/lexical-handler",
this);
parser.parse(is);
......@@ -11,37 +11,38 @@
<property file="dacapo.properties"/>
<property name="bm-name" value="batik"/>
<property name="batik.major.version" value="1.7"/>
<property name="bm-version" value="${batik.major.version}beta1"/>
<property name="batik.major.version" value="1.9"/>
<property name="bm-version" value="${batik.major.version}"/>
<property name="bm-src" value="${bm-name}-src-${bm-version}.zip"/>
<property name="bm-url" value="${apache.dl.url}/xmlgraphics/batik"/>
<property name="bm-url" value="${apache.mirror}/xmlgraphics/batik/source"/>
<import file="../common.xml"/>
<property name="batik-top" value="${bm-build-dir}/batik-${batik.major.version}"/>
<target name="unpack" depends="unzip"/>
<target name="unpack" depends="unzip, patch"/>
<target name="bm-build">
<exec executable="${ant.home}/bin/ant" dir="${batik-top}" failonerror="yes">
<arg line="all-jar"/>
<arg line="-Ddebug=on"/>
<env key="JAVA_HOME" value="${jdk16home}"/>
</exec>
</target>
<target name="jar" depends="deps">
<copy file="${batik-top}/batik-${batik.major.version}/lib/batik-all.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/batik-${batik.major.version}/lib/batik-all-1.9.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/build/crimson-1.1.3.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/xalan-2.6.0.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/xerces_2_5_0.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/xml-apis.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/xml-apis-ext.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/xalan-2.7.2.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/xercesImpl-2.9.1.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/xml-apis-1.3.04.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/xml-apis-ext-1.3.04.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/serializer-2.7.2.jar" todir="${bm-jars}"/>
<copy file="${batik-top}/lib/xmlgraphics-commons-2.2.jar" todir="${bm-jars}"/>
</target>
<target name="deps">
<copy file="${batik-top}/lib/js.jar" todir="${bm-deps-output-dir}"/>
<copy file="${batik-top}/lib/pdf-transcoder.jar" todir="${bm-deps-output-dir}"/>
<copy file="${batik-top}/lib/fop-transcoder-allinone-2.2.jar" todir="${bm-deps-output-dir}"/>
</target>
</project>
......@@ -15,8 +15,8 @@ import org.dacapo.harness.Benchmark;
import org.dacapo.parser.Config;
/**
* @date $Date: 2009-12-24 11:19:36 +1100 (Thu, 24 Dec 2009) $
* @id $Id: Batik.java 738 2009-12-24 00:19:36Z steveb-oss $
* date: $Date: 2009-12-24 11:19:36 +1100 (Thu, 24 Dec 2009) $
* id: $Id: Batik.java 738 2009-12-24 00:19:36Z steveb-oss $
*/
public class Batik extends Benchmark {
......
......@@ -23,7 +23,7 @@
<property name="bm-doc-dir" value="${bm-files}/doc"/>
<property name="bm-output-dir" value="${bm-files}/dist"/>
<property name="bm-output-doc-dir" value="${bm-files}/dist/doc/${bm-name}"/>
<property name="bm-output-harness-dir" value="${bm-files}/dist/harness"/>
<property name="bm-output-harness-dir" value="${bm-files}/dist"/>
<property name="bm-output-benchmark-dir" value="${bm-files}/dist/benchmark"/>
<property name="bm-deps-output-dir" value="${bm-files}/deps"/>
<property name="bm-data" value="${bm-files}/data"/>
......@@ -37,9 +37,9 @@
<property name="bm-dat" value="${bm-output-dir}/dat"/>
<property name="bm-cnf" value="${bm-output-dir}/cnf"/>
<property name="harness-classpath" value="harness/dist/harness"/>
<property name="harness-classpath" value="harness/dist/"/>
<target name="all" depends="source,svn-source,cvs-source,pre-build,build,complete"/>
<target name="all" depends="source,svn-source,cvs-source,hg-source,pre-build,build,complete"/>
<target name="incremental">
<property name="incremental-build" value="true"/>
......@@ -80,6 +80,19 @@
</antcall>
</target>
<target name="hg-source" if="bm-hg-url">
<condition property="fail-on-error" value="no" else="yes">
<isset property="bm-ignore-failure"/>
</condition>
<antcall target="check-hg-source">
<param name="target-dir" value="${bm-downloads}"/>
<param name="hg-url" value="${bm-hg-url}"/>
<param name="hg-revision" value="${bm-hg-revision}"/>
<param name="target-file" value="${bm-src}"/>
<param name="fail-on-error" value="${bm-fail-on-error}"/>
</antcall>
</target>
<target name="pre-build" depends="clean, init, unpack"/>
<target name="clean">
......@@ -162,7 +175,7 @@
</target>
<target name="complete">
<unjar src="${jar-target}" dest="${bm-output-dir}" overwrite="yes">
<unjar src="${build.target-jar}" dest="${bm-output-dir}" overwrite="yes">
<patternset>
<include name="doc/menu.html"/>
</patternset>
......@@ -175,7 +188,7 @@
</java>
<copy file="${temp.file}" tofile="${bm-output-dir}/doc/menu.html"/>
<delete file="${temp.file}"/>
<jar jarfile="${jar-target}" basedir="${bm-output-dir}" update="true"/>
<jar jarfile="${build.target-jar}" basedir="${bm-output-dir}" update="true"/>
</target>
</project>
......@@ -11,11 +11,11 @@
<property file="dacapo.properties"/>
<property name="bm-name" value="eclipse"/>
<property name="bm-version" value="3.5.1"/>
<property name="bm-drop" value="200909170800"/>
<property name="bm-url" value="http://archive.eclipse.org/eclipse/downloads/drops/R-${bm-version}-${bm-drop}"/>
<property name="bm-src" value="eclipse-SDK-${bm-version}-macosx-cocoa.tar.gz"/>
<property name="osgi-bootstrap-jar" value="org.eclipse.osgi_3.5.1.R35x_v20090827.jar"/>
<property name="bm-version" value="4.7.2"/>
<property name="bm-drop" value="201711300510"/>
<property name="bm-url" value="http://archive.eclipse.org/eclipse/downloads/drops4/R-${bm-version}-${bm-drop}"/>
<property name="bm-src" value="eclipse-SDK-${bm-version}-macosx-cocoa-x86_64.tar.gz"/>
<property name="osgi-bootstrap-jar" value="org.eclipse.osgi_3.12.50.v20170928-1321.jar"/>
<import file="../common.xml"/>
<property name="eclipse-top" value="${bm-build-dir}/eclipse"/>
......@@ -37,7 +37,14 @@
</antcall>
</target>
<target name="unpack" depends="untar"/>
<target name="moveEclipse">
<copy todir="${bm-build-dir}/eclipse/">
<fileset dir="${bm-build-dir}/Eclipse.app/Contents/Eclipse"/>
</copy>
<delete dir="${bm-build-dir}/Eclipse.app"/>
</target>
<target name="unpack" depends="untar, moveEclipse"/>
<target name="bm-build">
</target>
......@@ -49,20 +56,23 @@
<include name="eclipse-testing/eclipse-junit-tests-M*.zip"/>
</patternset>
</unzip>
<unzip dest="${bm-build-dir}/eclipse-testing/">
<fileset dir="${bm-build-dir}/eclipse-testing/">
<unzip dest="${bm-build-dir}/eclipse-testing/eclipse">
<fileset dir="${bm-build-dir}/eclipse-testing">
<include name="eclipse-junit-tests-M*.zip"/>
</fileset>
<patternset>
<include name="eclipse/plugins/org.eclipse.jdt.core.tests.performance*/${workspace-full}.zip"/>
<include name="plugins/org.eclipse.jdt.core.tests.binaries*"/>
</patternset>
</unzip>
<unzip dest="${bm-build-dir}/eclipse-testing/eclipse" >
<fileset dir="${bm-build-dir}/eclipse-testing/eclipse/plugins"/>
</unzip>
<unzip dest="${bm-build-dir}/eclipse-testing/workspace">
<fileset dir="${bm-build-dir}/eclipse-testing/eclipse">
<include name="**/${workspace-full}.zip"/>
</fileset>
</unzip>
<!--Create two complimentary zips for default and large workspaces-->
<!-- Create two complimentary zips for default and large workspaces -->
<zip destfile="${bm-build-dir}/${workspace-default}.zip" basedir="${bm-build-dir}/eclipse-testing/workspace">
<patternset includesfile="${bm-files}/bundles-default.lst"/>
</zip>
......@@ -83,17 +93,14 @@
<!-- build the eclipse plugin, which is necessary to fire up eclipse from within our harness -->
<target name="plugin" depends="unpack">
<exec executable="${ant.home}/bin/ant" dir="${bm-files}/plugin" failonerror="yes">
<arg line="clean"/>
<arg line="-Dplugin.destination=${bm-build-dir}/eclipse/plugins/"/>
<env key="JAVA_HOME" value="${jdk16home}"/>
</exec>