README
Last updated: October 26, 2005
Introduction
This readme describes how to
create performance charts and tables similar to those available on the
eclipse platform download pages using the stand-alone java program org.eclipse.test.performance.ui.Main
stored in this project. Although the program is currently geared to
generating results for the Eclipse Platform builds, the program may
work for you given the following conditions:
For purposes of this document, the term "fingerprint" will be used
to refer to a bar graph which represents the performance summary at a
global or component level for each test configuration.
The term "scenario
status table" will be used to refer to a table of scenarios with
green checks and red x indicators below a component fingerprint. This
table gives an overall view of the status for all scenarios for a
component for all configurations.
The term "scenario results" will be used to refer to a web page comparing current performance measurements to the baseline measurements for one scenario on one configuration. The web page displays the raw data for the current and baseline measurements and also displays line graphs showing the measured values over the course of builds between the current and baseline.
Requirements
Running the program
From a terminal or dos prompt,
Parameter descriptions:
- -baseline - baseline build id. A previous value used in the "build" key in the eclipse.perf.config system property or the eclipse.perf.assertAgainst property.
- -current - current build id. The value used in the "build" key in the eclipse.perf.config system property.
- -config - a comma-separated list of names of configurations for which to produce results. The values used should be the same as values specified for the "config" key in the eclipse.perf.config system property.
- -jvm -jvm description. The value used in the "jvm" key in the eclipse.perf.config system property.
- -output - path to an output directory for fingerprints and scenarios for each configuration specified in -config parameter. Fingerprint gifs and html files are generated in the directory specified, configuration specific subdirectories are created to store scenario results and line graphs.
Optional arguments:
- -config.properties - semi-colon separated list of: config, alternate config description. The value should be quoted if there are spaces in the value.
of builds which match any prefix in this list. When omitted, a magenta line is draw on the graph which represents the baseline value specified in the -baseline parameter.- -highlight - Comma-separated list of build Id prefixes(excluding 'I' and 'N'), which are used to add and highlight the most recent matching buildId on line graphs.
- -scenario.pattern - Scenario prefix pattern used to query performance results database. Can be used to generate results for a single component or subset of scenarios.
- -fingerprints or -scenarioresults - use one or the other to generate fingerprints or scenario results only. Not specifying either will execute both.
- -baseline.prefix - semi-colon separated list of: build id prefixes used in eclipse.perf.assertAgainst property or eclipse.perf.config system property. Values used to generate a second line graph representing changes in repeated baseline test run.
- -current.prefix - semi-colon separated list of: build id prefixes used in eclipse.perf.config system property. Values here used to override default of "N,I". Used to select build id's to include in main line graph.