If it were just a template library, it would be fine. But networking libraries? JSON parsers? The problem is much wider. If libfmt can have a couple .cc files so can you.
Sure, that's why I called it a language design issue.
Library users want generic and convenient interfaces (which arguably, boost does very well), there is just no way with modern C++ to keep the nice interfaces and at the same time put a significant amount of logic in a source file.
It is not surprising that boost choose convenience over tradition and fast compile speeds.