You know Linux has been getting soft and Gooey around the edges when there are Unix manuals for Linux users :-)
When I went from Slackware to FreeBSD in 1998, the only document you needed was INSTALL and the hardware compatibility list. In fact, FreeBSD was more posh than slack, it had a curses install and came with a luxurious handbook.
Speaking of which, I absolutely don't miss the days of taking a list of supported devices to the computer shop, and reading the fine print on boxes looking for chipset specs and firmware versions.
I'm running a production server (RAID setup, i7, so decent specs too) with multiple jails, each with their own ZFS filesystem. Honestly, I may be biased since I've been a FreeBSD fan since about '99, but it is really a sysadmin's dream. The system has been rock-stable for years. I'm currently running FreeBSD 8.1-RELEASE. Upgrading from 7 to 8 was not a problem, even from remote.
I'm a fulltime developer, so the less time I spend sysadmining on this system, the better it is for me. Upgrading packages, the kernel and even the jails is a quick job and can be set up to happen automatically.
Considering the high-quality and availability of Linux distributions these days, what are some of the advantages of using FreeBSD over Linux? More specifically, are there any competitive advantages that could be obtained from using FreeBSD over Linux that would outweigh the costs of deviating from the Linux standard on web servers?
At least for servers, FreeBSD is more polished than mainstream Linux (RedHat, Debian, and ancestors), if you ask me. The ports system keeps far more up to date than the typical Linux packaging system and the releases seem to be on a somewhat sane schedule. Of course, Linux is sort of this amalgamation of the kernel and whatever distribution you choose, so the ecosystem is fragmented and lacks the centralized command-and-control that FreeBSD has.
However, I still recommend and use Linux, mainly for support reasons. 7-8 years ago it was more of a toss-up as far as support, because Linux was far less established. These days it's a different ballgame. All the reasons to prefer FreeBSD (network & VM performance, stability, security, etc) are no longer clear advantages when compared to contemporary Linux. You are correct in your assertion that there simply isn't enough benefit to using FreeBSD to weigh out the lack of support both from a community and a hardware perspective.
Also, the Gentoo crowd seems to be similar, at least in spirit, to the FreeBSD folks. While they don't have the same deep integration of kernel & environment, they do have the same values when it comes to clean aesthetics and keeping up-to-date with packaged software.
> All the reasons to prefer FreeBSD (network & VM performance, stability, security, etc) are no longer clear advantages when compared to contemporary Linux.
Really? Why don't you think stability and security are important? What about consistency of interface between tools in a single system, and adherence to expectations of standard tools that make sense to the user (e.g., man pages instead of GNU Info pages, which are slowly creeping into Linux-based systems everywhere I look)?
I'm afraid I have to disagree with this estimation. In fact, the more time that passes, and the more the biggest Linux distributions are those that pander to people with expectations grown in the MS Windows ecosystem, the more the benefits of FreeBSD over Linux distributions grow. I suppose your mileage may vary, though.
> Also, the Gentoo crowd seems to be similar, at least in spirit, to the FreeBSD folks. While they don't have the same deep integration of kernel & environment, they do have the same values when it comes to clean aesthetics and keeping up-to-date with packaged software.
. . . but not when it comes to stability, from what I have seen.
I was a little unclear. My main point is that many of the things that made FreeBSD such a strong leader years ago are no longer weak points in Linux.
I agree with you though otherwise. From a cleanliness, usability, and aesthetics point of view, the state of Linux is pretty sad, especially when compared to FreeBSD. Linux could also learn a little bit about documentation from FreeBSD's excellent handbook.
EDIT: Let me also say that I'd really like to see the OpenSolaris devs jump to the FreeBSD team and help drive FreeBSD further.
If you have to ask that question and you're running web servers the BSD's are probably not right for you.
OS choice mainly comes down to preferences, some people like the way the BSD's do things, and some don't. There are some features that Linux has that the BSD's don't and vica-versa (like ZFS support on FreeBSD). But it's probably not going to matter on a web server.
Very true. At work, our sysadmin prefers Solaris so everything we run here is Solaris-based. I actually prefer Ubuntu Server for my personal projects since I prefer its Apache structure and easier package management. In the end, the results are the same.
It depends, apparently mysql for example poses real problems under FreeBSD under heavy load.
edit : to make it clear, Perlmonks.org for instance have performance problems because their host (which is also mine, BTW), pair.com, hosts the (heavy) mysql database backing the site under FreeBSD.
MySQL is not written in a very portable manner. I prefer PostgreSQL for a number of reasons, and the fact that its developers are concerned with portability is one of the more minor reasons.
There are a lot of reasons. The ports system has already been mentioned, and some people really like ZFS. More to the point, though, licensing differences ensure that more software (like ZFS) is legally compatible with FreeBSD than with any typical Linux+GNU distribution.
It has also been mentioned that FreeBSD is "a sysadmin's dream". A big part of the reason for this is its userland. The truth of the matter is that BSD tools are often much more consistent and less overly complex, and have better thought out basic functionality and interfaces, than GNU tools.
BSD Unix systems also resist the influence of the GNU project where many Linux distributions do not. For instance, in some distributions you will find far too many pieces of software that have nothing for man pages except a reference to how the real information is in a GNU Info page. The GNU project very much as a "not invented here" problem, where it not only wants to replace everything in the world -- even when it's "free software" -- with something weird and less usable that bears the GNU stamp on it. Meanwhile, the various BSD Unix systems insist on standardized tools and documentation that is easy to use, like man pages instead of Info pages.
Speaking of documentation, that's another good reason to use FreeBSD (or OpenBSD, for that matter). The big BSD Unix systems are among the best-documented OSes I have ever seen, and I have seen quite a few operating systems. Consider the completeness and helpfulness of the FreeBSD Handbook, which is actually one of the best pieces of documentation on the Web even for many Linux distributions, but is even more helpful for FreeBSD (of course).
Also . . . this is not comprehensive, but it might serve as a nice introduction to the whys and wherefores:
I would just like to point out that I'm pretty sure ports is the longest running joke in UNIX-like systems.
It always gets held up as an example of why FreeBSD is superior to Linux, and that's just insane. Its dependency resolution capabilities are awful, its third party repository capabilities non-existent, scripting it is an exercise in frustration, packaging for ports is easy but leaves me with feelings of horrible guilt because of the complete and utter lack of regard for the sanity of the end user, uninstallation is laughably weak particularly with regard to dependencies, and that's just the obvious deficiencies in the ports system. When you dig deeper, you find that the binary packages are universally unusable (often they don't even run because the libs they link against have been upgraded in the repository without updating the packages that depend on them), that you can't query ports for the actual state of the filesystem vs. what it thinks the state of the filesystem is because it simply doesn't know, and the list goes on and on. I find something new to hate about ports every single time I have to build something for it. apt-get and yum are vastly superior to ports in pretty much every single regard, though they are slightly harder to use for packaging software (but since ports doesn't support third party repositories in any reasonable fashion it doesn't matter, since you can't use the native package management on FreeBSD to distribute your applications anyway).
Sorry for the angry rant, but my level of disgust at seeing ports held up as an example of something good about FreeBSD is impossible to contain.
I generally think FreeBSD is a very fine operating system, with good documentation, and a very smart community. But, it's not because of ports.
> Its dependency resolution capabilities are awful
Funny. I had more problems with APT dependency resolution than with ports dependency resolution, probably by an order of magnitude, despite the fact that APT was the best package system I had encountered in Linux-land.
> its third party repository capabilities non-existent
I must admit that APT's third-party repository capabilities are pretty nice. On the other hand, I have not needed a third-party repository with FreeBSD yet, so I have no basis for comparison.
> scripting it is an exercise in frustration
Strange. I've scripted half a dozen different things for working with ports, because it was so easy to do, then subsequently found out that someone else had not only already done the same thing but done it better and committed it to ports. Your experience is obviously much different from mine.
> packaging for ports is easy but leaves me with feelings of horrible guilt because of the complete and utter lack of regard for the sanity of the end user
What's so insane about its (lack of) regard for the end user? Do you just think end users should never have to wait for a compile to finish? Is this an implied rehash of what you've already said? What exactly is the problem in this case?
> uninstallation is laughably weak particularly with regard to dependencies
Uninstallation appears about on par with APT from where I'm sitting.
> the binary packages are universally unusable
I don't bother using the binary packages, for the most part, but I know there are bunches of people who basically run their FreeBSD systems entirely off binary packages. I suspect they would disagree with your assessment.
> you can't query ports for the actual state of the filesystem vs. what it thinks the state of the filesystem is because it simply doesn't know
Example, or clearer explanation, please.
> since ports doesn't support third party repositories in any reasonable fashion it doesn't matter, since you can't use the native package management on FreeBSD to distribute your applications anyway
I find this difficult to believe, but what the hell -- I haven't bothered looking into it. I'll just accept what you say for argument's sake, for now.
> Sorry for the angry rant, but my level of disgust at seeing ports held up as an example of something good about FreeBSD is impossible to contain.
That's obvious. Too bad your disgust for it runs completely counter to my own experience.
"Funny. I had more problems with APT dependency resolution than with ports dependency resolution, probably by an order of magnitude, despite the fact that APT was the best package system I had encountered in Linux-land."
With ports it is entirely possible to install a package that does not work completely or at all due to missing dependencies. This is not something you ever need to think about with apt-get or yum. But on FreeBSD, I found it to be true of three or four packages out of a dozen or so that our software depends on (Apache, in particular, is really easy to screw up if you want any reasonable feature set; the ProFTPd port was broken for years). If you'd like an example, try installing the Apache port with all the major features that you'd expect in a virtual hosting environment, without intentionally installing all the pre-requisites in advance. You'll find that you've got a steaming pile of wreckage at the end of your nice long wait for it to build.
"Uninstallation appears about on par with APT from where I'm sitting."
It's trivially easy to remove a package required by other packages with ports. Meaning that it's entirely possible to break vast swaths of your system without warning.
"I don't bother using the binary packages, for the most part, but I know there are bunches of people who basically run their FreeBSD systems entirely off binary packages. I suspect they would disagree with your assessment."
I simply can't believe anyone could run a fully functional, and secure, FreeBSD system on nothing but binary packages. Many of them simply do not work once you bring the system up to date. The libraries move out from under the binaries, and they simply stop functioning. Either they'd be running an out of date, and thus insecure, system, or they're having to rebuild packages along the way to keep things working (and they'd have to test to know which stuff is broken, because ports and the pkg tools won't warn you when you're breaking stuff by updating software).
I also don't understand the mindset of anyone that chooses to wait hours for their system to be useful while they wait for a huge stack of ports to build. How little is your time worth to you?
"> you can't query ports for the actual state of the filesystem vs. what it thinks the state of the filesystem is because it simply doesn't know
Example, or clearer explanation, please."
rpm -V httpd
This command verifies the installation of httpd. Does what's running on the system match what the package contains? If not, it tells you which files have changed.
There is no equivalent for ports that I know of, and I'm pretty sure it would be deeply non-trivial to implement such a command because I don't think ports have a manifest or any sort of particular knowledge about what the payload of the package is. ports provides very little means of finding out what's already running on the system, which, from the perspective of someone trying to automate installation of a huge array of packages, while avoiding breaking any existing packages that the user may have pre-installed with custom options, is a pretty horrible experience.
"> since ports doesn't support third party repositories in any reasonable fashion it doesn't matter, since you can't use the native package management on FreeBSD to distribute your applications anyway
I find this difficult to believe, but what the hell -- I haven't bothered looking into it. I'll just accept what you say for argument's sake, for now."
Yeah, I find it difficult to believe, too. It's just stupid, but it's true. You can build a pkg and install it, but then there's no way to keep it up to date via native means (except requiring the user to download and install new pkgs over time, which is unacceptable when there are dozens of packages to deal with). You could also fudge your port into the ports tree, but that's undocumented, and the FreeBSD developers I talked to said it was a bad idea, and besides it still doesn't solve the problem of keeping your packages up to date using native means. You'd have to write a script or something that the user would need to run in addition to the usual ports management stuff to keep downloading and munging in your ports.
So, if you want to distribute a pile of applications and keep them up to date, you have to build your own update management system. Our products already have one (because it runs on hundreds of operating systems and versions, most of which don't have reasonable package management), but we strongly prefer to keep it native, since people already understand how to keep their system up to date using yum or apt-get, or whatever. We don't have that ability on FreeBSD.
"> Sorry for the angry rant, but my level of disgust at seeing ports held up as an example of something good about FreeBSD is impossible to contain.
That's obvious. Too bad your disgust for it runs completely counter to my own experience."
> With ports it is entirely possible to install a package that does not work completely or at all due to missing dependencies.
Meanwhile, I have discovered that with APT it is entirely possible for a failed install to hose up the entire package management system so that only "manually" fixing some broken dependency handling will allow me to use it again.
> It's trivially easy to remove a package required by other packages with ports. Meaning that it's entirely possible to break vast swaths of your system without warning.
Well, sure -- if you're a masochist. The same is true of Debian and Fedora, too, if you bypass the user-level tools and use the underlying utilities. If you use the user-level tools on FreeBSD, you get protections against doing that sort of thing, though. Let me guess: you always use "make deinstall" instead of pkg_deinstall. Right?
> I simply can't believe anyone could run a fully functional, and secure, FreeBSD system on nothing but binary packages.
I can't help what you are unwilling to believe.
> The libraries move out from under the binaries, and they simply stop functioning.
Are you installing libraries from ports and stuff that depends on the libraries from packages? I guess you could run into that problem under those circumstances.
> Either they'd be running an out of date, and thus insecure, system
You don't always have to update to run secure software. You only have to update when there are security updates.
> they'd have to test to know which stuff is broken, because ports and the pkg tools won't warn you when you're breaking stuff by updating software
I wouldn't know, because I haven't broken "stuff" by updating software the way you describe it. I have, however, had portupgrade warn me that if I installed something that required a different library version than another tool it would break something.
Hmm. I guess I do know, and you're wrong, in my experience.
> I also don't understand the mindset of anyone that chooses to wait hours for their system to be useful while they wait for a huge stack of ports to build. How little is your time worth to you?
I don't wait hours for my system to be useful while waiting for a huge stack of ports to build. I build stuff while I use the system. Y'know, I had the same question before I started using FreeBSD regularly, then discovered that it's not really an issue. Go figure.
> rpm -V httpd
>
> This command verifies the installation of httpd. Does what's running on the system match what the package contains? If not, it tells you which files have changed.
>
> There is no equivalent for ports that I know of, and I'm pretty sure it would be deeply non-trivial to implement such a command because I don't think ports have a manifest or any sort of particular knowledge about what the payload of the package is.
You must not be familiar with pkg_info, or the pkg-plist file that is part of every port for that matter. There's actually probably a few dozen ways to achieve the same thing, including using mtree to verify filesystem integrity.
Come to think of it, maybe you just aren't familiar with the ports system.
> Yeah, I find it difficult to believe, too. It's just stupid, but it's true. You can build a pkg and install it, but then there's no way to keep it up to date via native means (except requiring the user to download and install new pkgs over time, which is unacceptable when there are dozens of packages to deal with).
Yeah, yeah . . . like I said, I've never had to look into the use of third-party repositories, so I wouldn't know. I'm beginning to rethink taking your word for it, though, given your demonstrated lack of knowledge of basic ports management tools. In fact, a cursory look at the manpages for some ports management tools suggests that the ability to specify alternate sources for ports and/or packages is provided by these tools. I might look into this more.
> So, if you want to distribute a pile of applications and keep them up to date, you have to build your own update management system.
Even if the foregoing was true, this would not be true. There are gobs of cross-platform options available for software management, including even repurposed version control systems. You do not need to design your own software management system from scratch.
> people already understand how to keep their system up to date using yum or apt-get, or whatever. We don't have that ability on FreeBSD.
Have you tried submitting to ports?
> Give it time.
How much time does it take? I have been using FreeBSD for years already.
Obviously our experiences have been different. I'll readily admit that I'm not an expert with ports or apt-get (I've been a Red Hat, Fedora, CentOS user for about 15 years, so I'm deeply familiar with RPM, and know exactly what to expect there), and my entire experience with both has been building our installation script and updates management stuff for both platforms.
apt-get mostly just worked for everything I needed it to do. Yes, it has some occasional quirks, specifically when running it via scripts--there are some actions which are pretty much impossible to script. apt-get, for instance, won't replace a package with a different version to satisfy a dependency when run non-interactively, even if the package being replaced is not required by any other packages on the system. So, if you want to replace a package, you have to explicitly remove it first, and then install the replacement. This is weird, but it's not disastrous. I don't consider apt-get as reliable as yum/RPM for a number of reasons that I've discovered along the way, but it's miles above ports/pkg.
FreeBSD ports and pkg tools, on the other hand, have some really scary quirks. For example, installing a pkg non-interactively may fail, but return success. Or, it may return failure, but the package actually installed successfully. There had been a bug about it in the FreeBSD tracker about it for a couple of years when I ran into it...I don't know if it's been fixed by now. This was during the 7.0 release cycle. So, our installation verification stuff doesn't work on FreeBSD; we have to just say, "Installation probably completed OK, but we don't know for sure because the pkg tools don't always return accurate information about installation success or failure."
I've already mentioned the dependency issues I've run into. I can't believe I'm a unique and special flower in this regard, but maybe the dozen package we deal with and the exact options we need are uniquely broken when installing in a non-interactive fashion, and nobody else runs into them, ever. I certainly did. This aspect of supporting FreeBSD alone took a couple of weeks to work out, and the solution has never been to my satisfaction.
So, I'm happy to admit that my knowledge of ports is pretty miserable. But, the same can be said of my knowledge of apt-get...and the amount of effort that went into getting our installation script working on Debian vs. FreeBSD was dramatically different. And, the resulting reliability on Debian is dramatically higher than on FreeBSD. And, at the end of it all, the user can run "apt-get update; apt-get upgrade" and our packages get updated along with everything else. And, we have a nice UI in the product itself that can query the native package manager to display a list of available updates and such. All nicely integrated and comfortable for people who are already familiar with their OS. No such luck on FreeBSD. It's all different pieces that don't really fit comfortably together. Some are binary pkgs (because the install would take hours otherwise), some are ports, and some are from our own package format.
"> Either they'd be running an out of date, and thus insecure, system
You don't always have to update to run secure software. You only have to update when there are security updates."
So as long as no security updates happen, we're golden!
"> rpm -V httpd > > This command verifies the installation of httpd. Does what's running on the system match what the package contains? If not, it tells you which files have changed. > > There is no equivalent for ports that I know of, and I'm pretty sure it would be deeply non-trivial to implement such a command because I don't think ports have a manifest or any sort of particular knowledge about what the payload of the package is.
You must not be familiar with pkg_info, or the pkg-plist file that is part of every port for that matter. There's actually probably a few dozen ways to achieve the same thing, including using mtree to verify filesystem integrity."
I found pkg_info during my Trail of FreeBSD Tears, but I don't believe it tells you about the configuration of the installed port, does it? I don't remember exactly what information I needed and couldn't get, as it's been a couple of years since I was banging on it seriously, but I do recall there were details I couldn't get making it difficult to figure out the actual state of the system.
But, it's good to know that you can actually verify your packages and their contents. I'm not sure I believe it works as documented, since so few aspects of ports and pkg tools actually do in my experience, but in the absence of evidence to the contrary, let's just say it does. Point conceded.
"> Yeah, I find it difficult to believe, too. It's just stupid, but it's true. You can build a pkg and install it, but then there's no way to keep it up to date via native means (except requiring the user to download and install new pkgs over time, which is unacceptable when there are dozens of packages to deal with).
Yeah, yeah . . . like I said, I've never had to look into the use of third-party repositories, so I wouldn't know. I'm beginning to rethink taking your word for it, though, given your demonstrated lack of knowledge of basic ports management tools. In fact, a cursory look at the manpages for some ports management tools suggests that the ability to specify alternate sources for ports and/or packages is provided by these tools. I might look into this more."
If you figure it out, please let me know. I hate our current distribution and installation process on FreeBSD, and having the ability to use native packages with native update mechanisms would be great. I spoke to Colin Percival (created freebsd-update, and long-time FreeBSD developer) about it in the past, and he had no suggestions, beyond the already mentioned options.
"> people already understand how to keep their system up to date using yum or apt-get, or whatever. We don't have that ability on FreeBSD.
Have you tried submitting to ports?"
Some of the packages are commercial software, and the code is behind user:pass authentication on our servers. We also have to override the Apache and ProFTPd ports, because we need custom options (actually we're just building the port after modifying the configure options), though I believe ports handles this correctly and keeps our chosen build options through upgrades...I hope so, anyway, and I don't think we've gotten any tickets about it, so I guess it does, though our FreeBSD user base is pretty small (a few hundred vs. hundreds of thousands of CentOS installs). It's a pretty non-trivial endeavor. I believe Webmin and Virtualmin GPL, and some of the Virtualmin modules, already have ports. Which is cool, but we do need direct control over things for the commercial software.
"> So, if you want to distribute a pile of applications and keep them up to date, you have to build your own update management system.
Even if the foregoing was true, this would not be true. There are gobs of cross-platform options available for software management, including even repurposed version control systems. You do not need to design your own software management system from scratch."
As I mentioned, we've had our own cross-platform package manager built into our software for years. But, it's not native. That's the source of my complaint. I would like for users who are already familiar with the native means of keeping their system up to date can keep our software up to date, as well. I don't want more confusing update options for users to have to figure out. I want less. I want One True Package Manager on the system and I want it to be able to manage and update all the packages. Adding more software to an already large stack is exactly the opposite of what I want to do.
yum/rpm makes this trivially easy. apt-get makes it reasonably easy. FreeBSD makes it pretty much impossible as far as I can tell. But, hey, if you get it figured out, I'd be more than happy to be wrong. Supporting FreeBSD has been a pain in my ass for years now, and it'll continue to be so as long as installation is such a mess.
The tragedy of all this is that I really like FreeBSD. I think it's a fine OS with good docs and a really smart bunch of users (our FreeBSD users tend to be better educated about their systems than our CentOS users, on average; which makes sense, given that CentOS is the "standard" in hosting, and you'd have to go seek out FreeBSD). But, I just have had so much unbelievable pain and confusion and wasted time because of ports that it's rage-inducing.
> For example, installing a pkg non-interactively may fail, but return success. Or, it may return failure, but the package actually installed successfully.
News to me. Should I just take your word for it?
> There had been a bug about it in the FreeBSD tracker about it for a couple of years when I ran into it...I don't know if it's been fixed by now. This was during the 7.0 release cycle.
Do you have a link to that?
Anyway, bugs happen. For instance, some RPM signing functionality is broken [1]. There's also the fact that killing a simple yum info command can leave the rpmdb in an inconsistent state [2] -- and I'd call that a "really scary quirk" if it wasn't just a bug. Should I claim these are some kind of inherent failings of RPMs compared to FreeBSD ports, the way you claimed a bug was some kind of inherent failing of the FreeBSD ports system?
> I've already mentioned the dependency issues I've run into.
I've run into dependency issues with DEB and RPM based tools, too.
> And, at the end of it all, the user can run "apt-get update; apt-get upgrade" and our packages get updated along with everything else.
I asked around. It is absolutely possible to set up a third-party repository for FreeBSD ports. If you want to, you can even set up a second, completely separate ports tree, and a handful of lines of sh script can be used to as a wrapper around your ports management tool of choice to automate the process of selecting a ports tree from which to get what you need to manage a particular port if it isn't in the default tree -- if that's the route you want to take. I poked around in manpages for about three minutes before coming up with the basics for such a system, then decided before putting more effort into it to ask on the freebsd-questions list whether a third-party ports source could be used with standard ports management tools; the first response gave me an idea how to do this, and it pointed to exactly the sort of thing I had found in my manpage browsing.
Hire someone who uses (and likes) FreeBSD if you're having that much trouble finding what took me about three minutes to find.
> And, we have a nice UI in the product itself that can query the native package manager to display a list of available updates and such.
Getting that information out of the ports tree is trivial.
> I found pkg_info during my Trail of FreeBSD Tears, but I don't believe it tells you about the configuration of the installed port, does it? I don't remember exactly what information I needed and couldn't get, as it's been a couple of years since I was banging on it seriously, but I do recall there were details I couldn't get making it difficult to figure out the actual state of the system.
Well, sorry -- I can't tell you how to find something if you don't know what you need to find. I suspect it's there to be found, though, if it's not completely unreasonable.
> I hate our current distribution and installation process on FreeBSD, and having the ability to use native packages with native update mechanisms would be great.
I'd recommend starting with references to environment variables in the pkgtools.conf file if you want to get where I was going in short order. You can also get some hints from this response to my freebsd-questions query [3]. He talked a little more about mirroring than I had intended with my question, but a lot of the information is very generalizable.
> I want less. I want One True Package Manager on the system and I want it to be able to manage and update all the packages. Adding more software to an already large stack is exactly the opposite of what I want to do.
Admirable.
> yum/rpm makes this trivially easy. apt-get makes it reasonably easy. FreeBSD makes it pretty much impossible as far as I can tell. But, hey, if you get it figured out, I'd be more than happy to be wrong. Supporting FreeBSD has been a pain in my ass for years now, and it'll continue to be so as long as installation is such a mess.
If you're still having problems with this a year from now, you could always check back with me and see if my curiosity has driven me to look into it enough to write up a howto (or to find and test any that might have been written by others, since that would probably be preferable to duplicating effort).
> our FreeBSD users tend to be better educated about their systems than our CentOS users, on average; which makes sense, given that CentOS is the "standard" in hosting, and you'd have to go seek out FreeBSD
Have you considered throwing the question to your FreeBSD users to see if they can offer any help or suggestions to make things easier for themselves?
---
edit:
I don't know how long I'll be able to keep up with this discussion here. I'm not entirely clear on how long HN keeps comments open, and the interface for my comments list makes it difficult to keep up with older discussions. If you need to keep in further touch, try my contact page at blogstrapping [4].
So, as I mentioned, I tried going down the road of munging the ports tree and had it kinda working locally, but this was a stopping point for me:
"You have to be a bit careful with updating mechanisms if you customise your ports like this: portsnap for one is quite unfriendly to "foreign" files within the ports tree, but csup works fine."
All of our FreeBSD users told us they were using portsnap, and it's one of the things I'd talked to Colin about, I think (he wrote it). If we can't safely munge the ports tree in a way that's compatible with the tools our customers use, then we can't munge the ports tree. So, that option is right out. Again, if we break the usual process the user uses to update his system, it's not useful. If we're using multiple tools, multiple processes, and making the user do extra work or doing things that confuse them, we might as well just do it with our own package manager (we already have to maintain that repository for Solaris, Mac OS X, and other crappy systems with no package management to speak of).
The other options seemed to require more user involvement (i.e. switching between the standard and our repo manually), or adding an additional wrapper for the ports tools, thus requiring users to use a different command than usual. Further, if the repos are separate, and you switch between them, you no longer have any dependency resolution (even though I complain about it, at least there are some capabilities there...but only if all of ports is available). I guess we could run a complete mirror of the ports tree with our stuff munged in and all indexes and stuff regenerated, but that's just...too awful to think about. I also suspect our users would be unhappy with that solution.
In short, as far as I can tell, your link to the discussion on the FreeBSD mailing list confirms my own experience, which is: it doesn't work.
But, I could be misunderstanding something in that response. I'm quite clueless about a huge array of FreeBSD and ports questions. The answer could be staring me in the face, but I don't see it...just a rehash of all the problems I ran into when I went down this path a couple years ago.
Basically, we either break portsnap (or, more accurately, we'll get broken by portsnap) by munging into the existing ports tree, or we lose all dependency resolution by running a second separate repository.
Compare this to yum or apt-get, where a third party repository package is able to resolve from any other repository, including the standard ones, doesn't require any extra wrapper commands or knowledge on the part of the user, and setting it up on the users machine requires editing one configuration file and importing the relevant GPG keys.
"Have you considered throwing the question to your FreeBSD users to see if they can offer any help or suggestions to make things easier for themselves?"
Of course. The persistent ProFTPd problem I ran into finally got resolved by one of our users after a few weeks, actually. But, the third party repos problem was beyond all their kin, unfortunately. And, unless I'm mistaken, it's unresolved still, though I appreciate your, and the FreeBSD lists, attempts to help figure it out.
"Hire someone who uses (and likes) FreeBSD if you're having that much trouble finding what took me about three minutes to find."
I did. They didn't have any answers on the questions that remain. As I mentioned, I also hashed out some of this with Colin Percival (who, as far as I can tell, is at least six times smarter than I am, and knows a whole helluva lot more about FreeBSD and ports and portsnap than any one human ought).
While we're referencing, I couldn't dig up the bug about pkg_add return code issue, but I did find several people talking about pkgs simply not working:
"When a pkg just works, it’s a beautiful and fast thing.
The problem is when they don’t. That’s when ports shine."
"Back when I first started running FreeBSD, I completely destroyed my system installing the simplest stuff. Eventually, nothing would install. I do mean nothing. I had no idea what was going wrong, I just knew that I was frustrated. So I stopped using FreeBSD for a long time."
http://oscartheduck.wordpress.com/2007/07/25/on-ports-vs-pkg...
"Why choose one over the other? Well with packages sometimes the binary doesn't work. Either you happen to have a newer or older file than what the creator compiled it with or you changed something with your system and it doesn't match anymore with the author's compiled environment. However packages are fast to install. No need to break out gcc and compile source code."
http://returnbooleantrue.blogspot.com/2009/05/fixing-pkgadd-...
This was exactly my experience with binary packages. Some worked. Some didn't, and worse, the ones that didn't work provided no warning during installation that they would be inoperable once installed. Several of the packages we require had to be installed from ports because the binaries were non-functional. Making the install time for Virtualmin on FreeBSD a few hours (on a very small CPU-limited VPS, it can actually take days), vs. 10-20 minutes on other equivalently specced systems running CentOS or Debian (the small VPS case can take a couple of hours with these systems).
My concern about the return code bug is that it was persistent through multiple major revisions of FreeBSD. The first report of it was about 5.x, I seem to recall. It had existed and been discussed, but never corrected, over the span of a couple of years.
And, while the bugs you mention about RPM/yum sound annoying, they never caused me any trouble during my process of building our installation script. We certainly manage to tickle a lot of bugs, because we're doing so much heavy-lifting with the package managers on all these systems; and I've found loads of bugs on all of our supported systems (and I simply killed support for SUSE because their developers simply refused to fix package manager bugs that were showstoppers for us; closing them with unrelated "DUPE" statuses, and denying they were problems, at all; while Red Hat always fixed the bugs I reported, even if it took them a while). The difference is in how much it slows down our deployment, how much additional work it puts on me and our users, how reliably it performs its job as documented, etc. FreeBSD was, so far, our most complex target OS to support because of package manager issues. Technically, Solaris probably ought to hold that honor but we punted there, and took the easy way out, and said "screw the native package manager, it's shit anyway". I tried valiantly to use the FreeBSD package management tools as much as possible, which I paid for in a lot of time and frustration.
Another annoying fact is that most discussions about ports and problems therein result in redirection rather than admitting there are problems. Like the very common, "Just use ports" in response to any "binary packages do not work" kind of query; it seems to me that binaries ought to actually work consistently, or at least warn you during installation if they aren't going to work. This is what package management is for: to make sure the stuff you install works. If it doesn't, then the package manager is broken. I just ran into so many instances of packages not working with ports and binary packages, that my level of frustration is probably the highest it's ever been since starting Virtualmin (this company was designed from the get-go to be low frustration, as a response to my prior very stressful hardware company; so when it gets stressful, I get mad!).
"> And, we have a nice UI in the product itself that can query the native package manager to display a list of available updates and such.
Getting that information out of the ports tree is trivial."
But, since we're not in the ports tree, and can't be in the ports tree, it doesn't solve our problem. We could, and probably will, add support for ports to our updates UI (which could then merge the data of available updates with our wbm packages we already know about and display on FreeBSD), and so once installed, we could get close to an equally good experience for FreeBSD users, at least in the UI. The command line is still a mess, since updating our packages would be separate.
Anyway, I honestly do appreciate your going to the trouble to help figure it out, even if I'm pretty sure we're actually no closer to a solution that works.
> The GNU project very much as a "not invented here" problem, where it not only wants to replace everything in the world -- even when it's "free software" -- with something weird and less usable that bears the GNU stamp on it.
Oops. I lost track of how I was writing that sentence about halfway through. I meant to say this:
The GNU project very much as a "not invented here" problem, where it not only wants to replace everything in the world -- even when it's "free software" -- but chooses to do so with something weird and less usable that bears the GNU stamp on it, and works in a fully incompatible manner.
Also, (I haven't run a BSD for several years, but, from memory) one of the biggest philosophical differences is that *BSD runs a monolithic kernel whereas Linux runs a modular kernel. Might not sound like much, but it is very different mindset when it comes to security, user-land and such. Is that still the case?
All three of the major BSD derivatives have the ability to put parts of themselves in modules. They are all monolithic kernels, as is Linux.
Monolithic, in kernel speak, does not refer to having everything compiled in; it is more concerned about what is in kernel space (can do anything it wants with the hardware) and what is in user space (what is restricted at the hardware level to playing in its own memory). Monolithic kernels have pretty much everything in memory, and microkernels have pretty much everything in moderately-privileged userspace processes that talk to each other via message passing.
What koenigdavidmj said is true. The kind of "modular" that applies to the Linux kernel (and applies just as much to the FreeBSD kernel) is not the same kind of "modular" as you get with a microkernel OS like MINIX 3. All the major BSD Unix systems and any OS that uses a Linux kernel are all monolithic kernel OSes.
With BSD ports you compile the packages. The advantage is that you can select the compilation flags and choose dependencies and options. For instance, want to build emacs without X support? Just run WITHOUT_X11=yes make install clean. With apt you typically have packages like emacs-nox for special cases.
In my experience, FreeBSD's greatest asset doesn't fit well on a feature list. It's that it has a different attitude towards change than Linux.
Linux is a diverse ecosystem, with tons of experiments driving it forward. Different package managers, different kernel patches, different userland configurations, different means of administration... and that's a great thing.
FreeBSD is not diverse or experimental. FreeBSD is a deliberately moving beast, that adopts technologies and ideas in a more methodical manner, often after other OS's (OpenBSD / NetBSD / DragonflyBSD / Solaris / Linux / OS X) have proved that the ideas are solid, and lasting, and then it integrates them carefully, so it still looks like FreeBSD (but with some new capabilities, or better performance, or whatever), rather than having drastically different administration mechanisms, or performance characteristics, or anything else.
Personally, I like this stability. I like that it's a platform that once learned, can be largely forgotten about. But that said, it does require learning first, and while it's powerful, it's not shiny or sexy. But even excluding the man pages and documentation, zfs, the pf firewall, and many other things I like about FreeBSD, this is the winner for me. I just love that it doesn't change unless the change is a large win.
The common commands page[0] lists a few translations for package managers. It would be great if someone could add a FreeBSD column to the much more voluminous Pacman Rosetta page[1] (which is intended for Arch users, but provides a helpful matrix for all the common Linux distros).
You know Linux has been getting soft and Gooey around the edges when there are Unix manuals for Linux users :-)
When I went from Slackware to FreeBSD in 1998, the only document you needed was INSTALL and the hardware compatibility list. In fact, FreeBSD was more posh than slack, it had a curses install and came with a luxurious handbook.
Speaking of which, I absolutely don't miss the days of taking a list of supported devices to the computer shop, and reading the fine print on boxes looking for chipset specs and firmware versions.