+ + +

Clang - Performance

+ + +

This page tracks the compile time performance of Clang on two +interesting benchmarks:

Sketch: The Objective-C example application shipped on + Mac OS X as part of Xcode. Sketch is indicative of a + "typical" Objective-C app. The source itself has a relatively + small amount of code (~7,500 lines of source code), but it relies + on the extensive Cocoa APIs to build its functionality. Like many + Objective-C applications, it includes + Cocoa/Cocoa.h in all of its source files, which represents a + significant stress test of the front-end's performance on lexing, + preprocessing, parsing, and syntax analysis.
176.gcc: This is the gcc-2.7.2.2 code base as present in + SPECINT 2000. In contrast to Sketch, 176.gcc consists of a + large amount of C source code (~220,000 lines) with few system + dependencies. This stresses the back-end's performance on generating + assembly code and debug information.

+ + +

Experiments

+ + +

Measurements are done by serially processing each file in the +respective benchmark, using Clang, gcc, and llvm-gcc as compilers. In +order to track the performance of various subsystems the timings have +been broken down into separate stages where possible:

+ +

-Eonly: This option runs the preprocessor but does not + perform any output. For gcc and llvm-gcc, the -MM option is used + as a rough equivalent to this step.
-parse-noop: This option runs the parser on the input, + but without semantic analysis or any output. gcc and llvm-gcc have + no equivalent for this option.
-fsyntax-only: This option runs the parser with semantic + analysis.
-emit-llvm -O0: For Clang and llvm-gcc, this option + converts to the LLVM intermediate representation but doesn't + generate native code.
-S -O0: Perform actual code generation to produce a + native assembler file.
-S -O0 -g: This adds emission of debug information to + the assembly output.

+ +

This set of stages is chosen to be approximately additive, that is +each subsequent stage simply adds some additional processing. The +timings measure the delta of the given stage from the previous +one. For example, the timings for -fsyntax-only below show +the difference of running with -fsyntax-only versus running +with -parse-noop (for clang) or -MM with gcc and +llvm-gcc. This amounts to a fairly accurate measure of only the time +to perform semantic analysis (and parsing, in the case of gcc and llvm-gcc).

+ +

These timings are chosen to break down the compilation process for +clang as much as possible. The graphs below show these numbers +combined so that it is easy to see how the time for a particular task +is divided among various components. For example, -S -O0 +includes the time of -fsyntax-only and -emit-llvm -O0.

+ +

Note that we already know that the LLVM optimizers are substantially (30-40%) +faster than the GCC optimizers at a given -O level, so we only focus on -O0 +compile time here.

+ + +

Timing Results

+ + + +

2008-10-31

+ + +

Sketch

+ +

This shows Clang's substantial performance improvements in +preprocessing and semantic analysis; over 90% faster on +-fsyntax-only. As expected, time spent in code generation for this +benchmark is relatively small. One caveat, Clang's debug information +generation for Objective-C is very incomplete; this means the -S +-O0 -g numbers are unfair since Clang is generating substantially +less output.

+ +

This chart also shows the effect of using precompiled headers (PCH) +on compiler time. gcc and llvm-gcc see a large performance improvement +with PCH; about 4x in wall time. Unfortunately, Clang does not yet +have an implementation of PCH-style optimizations, but we are actively +working to address this.

+ +

176.gcc

+ +

Unlike the Sketch timings, compilation of 176.gcc +involves a large amount of code generation. The time spent in Clang's +LLVM IR generation and code generation is on par with gcc's code +generation time but the improved parsing & semantic analysis +performance means Clang still comes in at ~29% faster versus gcc +on -S -O0 -g and ~20% faster versus llvm-gcc.

+ +

These numbers indicate that Clang still has room for improvement in +several areas, notably our LLVM IR generation is significantly slower +than that of llvm-gcc, and both Clang and llvm-gcc incur a +significantly higher cost for adding debugging information compared to +gcc.

+ +