What makes you assume the AI companies actually want to create a super intelligence they can’t control. Altman has stated as much. Musk definitely wants to remain in power.
Not yet, I agree, but who is to say they couldn't?
Limiting life to cell based biology is a somewhat lousy definition by the only example we know. I prefer the generalized definition in "What is Life?" by Erwin Schrödinger which currently draws the same line (at cellular biology) but could accommodate other forms of life too.
I sometimes wonder about what our governments would do if one of the businesses in their jurisdictions were to achieve AGI or other such destabilizing technology. If it were truly disruptive, why would these governments respect the ownership of such property and not seize it - toward whatever end authority desires. These businesses have little defense against that and simply trust that government will protect their operations. Their only defense is lobbying.
AGI is end game scenario. That is "winning". If a business wins it, then the government may not remain subservient to it no matter what free market conditions it had preserved beforehand, as long as it has the power to act.
