As a previous compiler developer, its a pretty bad idea to look at the final lines of code to determine its cost.
Typically in compiler development, there's massive R&D time, many hours spent analyzing memory & CPU patterns to figure out which code to emit. You also take a look at disassembly of generated code and nowdays perform data-mining to figure out which patterns of optimize for (typically on non-kernel or hotspot-heavy type workloads).
The end result of alot of investigation can be few lines of code which have to be absolutely robust and production ready at the time of release under standard configurations (i.e -O1, -O2 etc...)
Its a costly piece of engineering which is why some companies still charge money for it.
I'd say the article's estimates are accurate for a simple compiler that emits results at say -O0 or -O1 level of efficiency, but anything higher will start to require more and more resources with diminishing gains over time. But this can be said about most things in engineering.
> LLVM is smaller at 1.6 million lines, but is newer, supports only C and C++ by default
Unless I'm misunderstanding "by default", this isn't true. LLVM is language agnostic and supports many languages - even the primary C-like frontend Clang supports more than just C/C++.
We have worked with LLVM a lot for own own compilers and DSPs.
LLVM Intermediate representation in practice is C++, AST is C++. LLVM do not support many languages,many languages support LLVM, which is different.
That is many people, or languages' designers, have made the necessary work to make their languages compatible with LLVM IR. If the language is similar to C++, this is a simple task, if it is very different, it is very hard.
LLVM does not support other languages, in fact, codebase changes a lot making maintenance painful.
Specifically, in my limited experience LLVM IR resembles typical hardware rather than C++. Modeling the hardware is a design goal for C++, of course, so anything hardware-like must be C++-like. Can you describe some ways in which IR resembles C++ that are not plausible ways to resemble hardware? I'm really curious.
There exists no such thing as "language agnostic". Every intermediate representation contains properties (very often implicitly) that are specific at least to a class of programming languages.
Those languages are however supported mostly by (accidentally or deliberately) being similar to C/C++. Not only does the IR need to be able to express certain constraints of the language semantics (IIRC, there have already been changes to better accommodate Rust), but the kinds of IR patterns the compiler frontend generates might not get optimized if they are too un-C-like.
Now, before GHC generates assembly code (abiding by the calling conventions previously described), it also needs to optimize it. It optimizes a form called "Cmm", which is like a very low level compiler language for performing optimizations. This language does include functions.
When you use the LLVM backend, GHC translates every Cmm function to an LLVM function. In order for everything to work out, we patched LLVM so that functions can have an annotation saying they follow the GHC calling convention.
...
What this means is, we have completely side stepped LLVM's support for GC.
So the support mostly comes from the Haskell side, by compiling to the C-like Cmm (I think that name comes from C--, the idea being a language between assembly and C in abstraction level) and patching or otherwise working around the part where LLVM's feature set wasn't appropriate.
If the intermediate representation is no more language-specific than x86, then it isn't really the compiler that's language-specific: it's modern computer architecture.
That is only necessary for single objects that are bigger than one page which, we dont currently do (but should). Arrays are no problem since we check the bounds, only raw pointers cause problems.
The GCC addons (gcc-fortran, gcc-cobal, and gcc-go) just convert the existing language into C that is just ingested into the standard GCC. It was only around 2008 that gcc-c++ stopped doing the internal translation/name mangling dance and gained native first party support [1].
Much like how the older versions of GCC used to just produce ASM files to be ingested into GAS to create the binary objects.
Yeah, I thought that part of the point of the LLVM project was that front end language compilers (entirely language agnostic) could target non-platform specific LLVM IR code, and then let the backend LLVM compiler compile to multiple backend targets. It simplifies front-end compiler development by having a single IR target over multiple native target.
This was between 1989 (when we started in my living room) and 1999 when I left. By 1999 we had about 160-170 developers working on gcc, gdb, binutils, glibc, Cygwin etc. The money was a mixture of 1> ports (typically paid for by the manufacturer), 2> maintenance (mostly paid for by the corporate end user -- for $100K/year we'd fix your bug whether we spent $100 to make a trivial fix or $300K on a major problem) 3> maintenance on very old versions that people had to freeze on for one reason or another and 4> the public tree (we wouldn't work on non-free software, apart from a few abortive experiments after I had left).
Making sure everything we wrote was compatible with the public tree was expensive. In the end, one of the last things I did saved a lot of money and improved code quality for everyone: I forked gcc from the FSF (and made the egcs tre the de facto, and eventually de jure version).
Nowadays forking is considered routine, and even beneficial, and projects have steering committees that are responsible for master releases. At that time, forks were considered a tragedy, and it took me over 6 months of discussion with various people to build a reasonable consensus (you can see the people's name at the bottom of the letter I wrote announcing it ( https://gcc.gnu.org/news/announcement.html ). It's been a good model. Note that RMS's name is not on the letter -- he was furious and was sure this would destroy the FSF. Instead it has strengthened it.
By the way, to compare these numbers: Red Hat went public with, AFAIK less than 10 engineers and when Cygnus and Red Hat merged, though both companies had about 200 people, Cygnus had over 160 engineers while Red hat had a couple of dozen. Of course we had a lot more revenue, and in the years since RHAT has invested heavily in free software development. But their company value at that time was something like 3X ours -- a very valuable lesson.
I remember writing code for Psion (later Symbian) Epoc32 in the late 90s and early thousands. It was a big deal that Cygnus had done the work to make gcc support ARM because the vendor compiler was (IIRC) some weird combiniation of expensive and not very good.
We now assume ARM as being a default necessary platform to support, 20 years ago it really wasn’t clear
I find it odd that they did an LLVM port and even another blog about dealing with 16bit char types, but none of the target specific changes are going upstream. I can understand a company not wanting to help their competitors, but it sounds like they're still going to try to get the 16bit char changes pushed upstream, which is likely one of the more useful pieces for a competing DSP vendor with a different instruction set.
In the scientific / numeric space it’s not unheard of to buy Intel Fortran/C compilers for performance reasons.
Also, it’s been a few years since I touched IBM’s POWER hardware with AIX, but I’m pretty sure XLC and whatever the Fortran compilers were didn’t come free.
I wish Intel would make their compiler free, or work harder on integrating their optimizations into free compilers. They can't be making much money off it, and it just feels counterproductive to charge customers extra to get the best performance from their own hardware.
Anecdotally, changes to ICC are evaluated (by management) on two metrics: improvement on Intel CPU, but also improvement on Intel CPU minus improvement on AMD CPU. The end result is similar to RUN_SLOW_ON_AMD flag.
If you think this cheesy, consider this: you are a manager of Intel compiler team. Someone implemented an optimization that improves performance by 1% on Intel, but 10% on AMD. Do you think it is a good change? The change will be no-brainer on GCC, but consideration for ICC is necessarily different.
My organization uses the Intel Fortran compiler. I did a few rough benchmarks at one point and found it produced code nearly 3x faster than the same code in gfortran.
I hear this a lot, but if it's with equivalent optimizations it's quite pathological and should be reported as a bug; it's certainly not generally the case. They are typically similar at the 10-20% level in my experience (not always in ifort's favour). There are several significant HPC codes which recommend GCC where it matters. You also need to consider reliability; GCC is surprisingly better in my experience -- I've quite often solved users' problems with "just try GCC".
There used to be a published set of results from Polyhedron -- presumably to help to sell proprietary compilers -- which had a geometric mean favouring ifort by ~20%, if I remember right. However, the last I saw didn't use the latest GCC, required the same flags for each case (which didn't include profile-directed, for instance), and the cases with the biggest differences actually bottlenecked on libraries, not generated code. gfortran is infinitely faster on most architectures, of course.
Most benchmark results I see aren't useful because they don't supply the parameters needed to be reproducible and they don't have profile information to allow you to understand, and maybe improve, them.
I found that results from the Polyhedron benchmarks are online again <https://www.fortran.uk/fortran-compiler-comparisons/polyhedr.... Note that they compare gfortran from 2015 and ifort from 2017. I don't know how representative they are of the part of the workload on typical HPC systems for which the compiler might be important.
I suspect the exact speedup depends a lot on the workload, but the last time I supported scientists on HPC, they reported pretty substantial performance gains with ifort over gfortran as well. But then Intel has deeper pockets to pay for this kind of thing than the FSF.
I think I also recall something about gfortran deliberately choosing some more conservative optimization strategies, which could also explain some differences. Or I could be misremembering absolutely everything :-)
Much large scale computation on a typical general HPC system is quite insensitive to the compiler, with most time spent in maths libraries and/or MPI. Most scientists compiling code don't know anything about performance engineering, or even measuring things, unfortunately.
GCC does correct optimization by default (modulo bugs). ifort's default set of flags on GNU/Linux at least used to be non-standard-conforming (maybe fast-math?). Comparisons need to use equivalent flags for gfortran. I don't have the ones I once noted to hand, but they'd include -Ofast -funroll-loops -march=native. The ifort default also generates static binaries, which require static libraries and which you typically don't want so that you can do profiling, for instance.
Maybe. For what it's worth, OpenBLAS is at least as fast on all AMD64 micro-architectures before AVX512 that I've had access to. That's for DGEMM, which is what normally matters; MKL may well have optimized more kernels than OpenBLAS has. Unfortunately, many cling to performance myths in the face of measurements.
Yeah, my last experience with paid MKL libs / Intel compilers was some years ago now—before I ever remember seeing anything about OpenBLAS. I know OpenBLAS has been a big performance boon.
And to be clear, I’d love to see Free and Open Source options be the standard for HPC. It sounds like we may be much closer to that state today than we were years ago when last I worked with people highly concerned about performance in this area. That’s great news.
> Now I want to write compilers and developer tools, but there's no money to be made.
It's sad. One the one hand it's great that we have access to so many amazing compilers for free. But now there's a monopoly of compilers - and by extension, programming languages - by large corporations. They're all kind of the same thing too - vaguely C-like statically typed ALGOL languages with OO and FP features bolted on top. Swift/Kotlin/C#/Java/Typescript/Go, it's all much the same.
I don't think anything novel (a smalltalk, prolog, haskell or scheme) could get any mindshare today, unless a big company lost their mind and tried to push something weird.
Red Hat tend to hide bug reports on their commercial (free software) compilers in bugzilla, unfortunately; I don't know what Adacore, for instance, or other commercial providers do, and such bug reports will be only a small part of it. You could obviously study the defects in released versions of GCC on which commercialized versions are based, though, to the extent they have PRs.
A great benefit of GCC was always enabling learning and research, though there are more opportunities these days.
Posts like this make me think of this: https://github.com/oriansj/M2-Planet
A C compiler with support for inline assembly, gotos, structs and weighs under 2000lines and is self hosting.
I was hoping this was an analysis on how much time and money is spent while waiting for n number of lines of code to compile by various compilers. I find waiting for things to build, deploy, install dependencies, etc as the most mind numbing aspects of the field. I can hardly stand it.
I wonder, are there languages that are cheaper yet have similar features?
For example, how much is rust? python?
It's weird how a compiler can cost so much, yet it can still produce unsafe code because the language is not safe by design. In that optic, rust might save a lot of money.
You cannot produce a Rust or a Python compiler without a working C/C++ compiler first anyway, so the point is slightly moot. If there exists a C/C++ compiler for the platform you want to build for, just build Python for it and you're good to go.
C/C++ compilers are different from other compilers. They are forced to support all kinds of complicated, too-many-cooks-spoil-the-broth, design-by-committee standards. If you just wanted to do another portable assembler, your new compiler would take just a fraction of the effort.
Typically in compiler development, there's massive R&D time, many hours spent analyzing memory & CPU patterns to figure out which code to emit. You also take a look at disassembly of generated code and nowdays perform data-mining to figure out which patterns of optimize for (typically on non-kernel or hotspot-heavy type workloads).
The end result of alot of investigation can be few lines of code which have to be absolutely robust and production ready at the time of release under standard configurations (i.e -O1, -O2 etc...)
Its a costly piece of engineering which is why some companies still charge money for it.
I'd say the article's estimates are accurate for a simple compiler that emits results at say -O0 or -O1 level of efficiency, but anything higher will start to require more and more resources with diminishing gains over time. But this can be said about most things in engineering.