I think part of the parent comment's point was that Swift does/will have a lot of users. Right now those 10,000 Go questions cover a lot of server related issues; the 37,000 Swift questions are almost completely iOS questions.
If I started writing an HTTP server in Swift the day it comes out I won't find much help but I'll have a mountain of unrelated answers to filter. Essentially Swift's adoption for iOS apps doesn't help and might hurt efforts to use it elsewhere.
Look at Object-C: extremely common writing software for Apple; almost non-existent elsewhere, despite the fact that it has never been limited to Apple.
Objective-C never caught on outside of the Apple/NeXT ecosystem because its biggest advantages were in the application frameworks, not the language itself. By the time iOS made Objective-C popular there were entrenched alternatives in the C++, Java, and .NET ecosystems. Swift is facing the same challenges, but against a newer generation of competitors that are much less established. It's probably still an uphill battle for Swift to gain outside adoption, but it's not climbing a cliff.
Not to mention, Swift the language itself, which is basically Rust with arguably an easier memory management model (all ARC) and Scala-like pragmatism (an OOP model built-in). Coming from ObjC, developers are showing a high level of interest in Swift. I might look it up when it comes out for Linux!
I have carefully compared the two languages and I do not see more similarity than between Go and Swift or any other language. You have yourself cited the different memory management models, and this is the main characteristic of Rust. The sole common feature is the use of LLVM.
If I started writing an HTTP server in Swift the day it comes out I won't find much help but I'll have a mountain of unrelated answers to filter. Essentially Swift's adoption for iOS apps doesn't help and might hurt efforts to use it elsewhere.
Look at Object-C: extremely common writing software for Apple; almost non-existent elsewhere, despite the fact that it has never been limited to Apple.