I had an idea as I was reading this. Open source seems counterintuitive because it gives away code, which people assume is valuable. As the article says, "How can you build a business by giving away the store?" But what if code isn't that valuable?
More precisely, what if code isn't worth very much when separated from the programmers who wrote it? This notion explains a lot. It explains why source code theft is almost unheard of (http://news.ycombinator.com/item?id=36551) - except, of course, programmers taking copies of their own code with them when they leave a company. It explains why codebases decay so quickly when they're handed off for maintenance, and why rewrites almost always fail when they're not done by the original team. It explains why people don't really buy or sell source code as such.
It also matches the experience of working programmers, which is: most code sucks; other people's code really sucks ("it's all shit", an oldtimer once told me); and it's often easier to write code than to read it.
The best description I've found of this idea is Peter Naur's (http://news.ycombinator.com/item?id=121291). Naur puzzled over cases where, even when code was well written and well documented, other programmers (not members of the original team) screwed it up. His explanation for this is that the program is not the code. Rather, the program is a set of mental constructs (Naur uses the word "theory") that live in the minds of the programmers writing it. Code is merely the canonical written expression of the program, and it's a lossy one: given only the code, you can't reconstruct the theory. Since the theory exists in the minds of the team, if you lose the team, you lose the theory, and with it most of the value.
Viewed this way, the economics of open source vis-à-vis closed source are not nearly so counterintuitive. If code isn't that valuable, then open source isn't "giving away the store" at all. And if what is valuable is the theory of the program in Naur's sense, then open source adds value by extending the theory to a larger base of programmers, something that closed source has tried and failed to do for years.
If open source is value-creating, it's not an economic paradox. It only looks that way because people (especially non-programmers) have a mistaken model of software value. It's not surprising that it would take time for business models and infrastructure to catch up.
While I don't necessarily disagree with you, I personally love working with other people's code. Better or worse than what I would have written, at least its a starting point. Plus it puts a new perspective on a problem that I wouldn't have gotten without tinkering with the open source solutions first.
Actually, I have to say something else about this. You mention working with other people's code in an open source context. That relates to my original point - there seems to be something about open source that changes the dynamic I was describing and that relates to the value of software. After I noticed this detail in your comment I asked myself, does it feel different to work with other people's code when that code is open source, and (for me at least) the answer is definitely yes, it feels better. It would be fascinating to know why.
You make a very valid point, one which I think is consistent with what I was saying, though perhaps not obviously so.
I'd make a distinction between large production codebases that have been around for a while (let's call that "legacy code") and code that illustrates how to do a particular task (let's call that "sample code"). These are not mutually exclusive; legacy code can be mined for good sample code, and the better systems are rich sources of it. When you're trying to figure out how to do "particular thing X" there's no better way than to look at working code that does X.
When Naur talks about the theory of a program, though, he's not talking about particular thing X, but about grokking the design and structure of an entire system. That's a hard thing to do for someone who doesn't have contact with the original programmer(s). You don't need that kind of knowledge for tinkering or bug fixing, but it's hard to pull off major design transformations without it.
So what I was saying applies to legacy code, but not sample code. This matches experience, I think. When working with legacy systems, I've gotten a great deal of mileage out of asking myself "how did they do blah" and then looking to the code for blah to see exactly how they did it. But this only works for relatively constrained problems like "how did they scale that font" or "where did they get that data". It doesn't work if "blah" equals "producing the whole system".
Web apps illustrate this. We all inspect the code of other web apps to figure out specific techniques. That this is so easy to do is of course one of the biggest reasons for the rise of the web in the first place. But it's orders of magnitude harder to figure out the design of an entire system. I've actually tried - programmatically downloaded as much js and html as I could possibly scrape from another app, to learn whatever I could about how it was put together - and it was a total waste of time.
I'm curious to hear if this distinction is relevant to what you were saying.
It's absolutely relevant, and I would generally agree with you.
There is one exception though. If the system is surrounded by a well designed testing framework, it can be overhauled piece by piece as necessary, no system rewrite required. Test code illustrates what the original programmer expected, and enables a system to be extended without necessarily understanding the theory behind it in its entirety. Of course, most legacy systems don't have good testing frameworks in place these days, so it might take a few years to see if this exception actually holds true or if the testing frameworks tend to get scrapped with the system.
More precisely, what if code isn't worth very much when separated from the programmers who wrote it? This notion explains a lot. It explains why source code theft is almost unheard of (http://news.ycombinator.com/item?id=36551) - except, of course, programmers taking copies of their own code with them when they leave a company. It explains why codebases decay so quickly when they're handed off for maintenance, and why rewrites almost always fail when they're not done by the original team. It explains why people don't really buy or sell source code as such.
It also matches the experience of working programmers, which is: most code sucks; other people's code really sucks ("it's all shit", an oldtimer once told me); and it's often easier to write code than to read it.
The best description I've found of this idea is Peter Naur's (http://news.ycombinator.com/item?id=121291). Naur puzzled over cases where, even when code was well written and well documented, other programmers (not members of the original team) screwed it up. His explanation for this is that the program is not the code. Rather, the program is a set of mental constructs (Naur uses the word "theory") that live in the minds of the programmers writing it. Code is merely the canonical written expression of the program, and it's a lossy one: given only the code, you can't reconstruct the theory. Since the theory exists in the minds of the team, if you lose the team, you lose the theory, and with it most of the value.
Viewed this way, the economics of open source vis-à-vis closed source are not nearly so counterintuitive. If code isn't that valuable, then open source isn't "giving away the store" at all. And if what is valuable is the theory of the program in Naur's sense, then open source adds value by extending the theory to a larger base of programmers, something that closed source has tried and failed to do for years.
If open source is value-creating, it's not an economic paradox. It only looks that way because people (especially non-programmers) have a mistaken model of software value. It's not surprising that it would take time for business models and infrastructure to catch up.