Except they wouldn't. It took Apple a decade to up their silicon to a level where it is now. For most of that time, their activity was a money sink for them. And even today, even with some of the best talent in IC design one can get in USA (PA Semi and its veteran designers from HP, Sun, Freescale, and whatever else was remaining of American semi industry), their chips are still half outsourced, 70% external IP if you remove SRAM from calculations.
Apple is company with an own silicon, but it will never be a true microelectronics company.
Google's TPUs are another examplary case. Goog did spare no money to get into the market first and secure a tech/platform lockdown. Yet, what took them at least 5 years, and extremely expensive to make reticule limit chips, was bested by a no-name Chinese fabless that blew TPUs out of the water on power/cost/performance ratio.
All American dotcoms that are actively trying to get into hardware are oblivious to the fact that there is an inherent difference in between a hardcore engineering company to a company whose topmost technical expertise is underhanded web programming.
I won't be so certain. An ASIC/FPGA chip that accelerates specific Database operation is already a reality. The problem is the cost effect ratio.
BTW, cloud provider already making tons of customized hardware themselves. And talking like Amazon/Google/Microsoft only have talents for web programming is short sighted and simply not sure.
>And talking like Amazon/Google/Microsoft only have talents for web programming is short sighted and simply not sure.
Well, I am saying that they do obviously spend an ennormous effort trying to do so. To better formulate what I said, the fact that they did acquire some hardware/semi expertise in house, does not mean that this expertise drives them as a company.
If tomorrow 2 engineers, one from semi side, and one fromwebdev, will knock at the door of CEO with "I found a way to do things A and B 250 times better, but you have to scrap half of your business plan," a webdev will be heard, and semi guys not.
Btw, the chip Amazon showed, was from a company they acquired solely for that - to not to pay an arm and a leg for high end switching chips.
And about OEM servers - it is surprising that big dotcoms are late comers to the party, and were relying on off the shelf brand hardware to the very last moment.
Big hosting providers from outside of dotcom ecosystems were relying on direct OEM orders and custom built DCs for more than a decade. I do remember selling Atom based single board computers stuffed into U1s and Intel core 2 duo systems with soldered on memory and cpus to budget web hosting guys back when I worked as a trainee in a trade company back in 2007-2009.
> "If tomorrow 2 engineers, one from semi side, and one fromwebdev, will knock at the door of CEO with "I found a way to do things A and B 250 times better, but you have to scrap half of your business plan," a webdev will be heard, and semi guys not."
What are you basing this assumption on? Have you worked for one of these companies? Do you know anyone that does?
Also, regarding Microsoft, any suggestion that they're focused on acquiring web devs is clearly short sighted. If you want a better idea of its priorities, I'd suggest taking a look at which sectors it earns its main revenue in, as well as taking a look at the work being done at Microsoft Research.
It takes a giant effort for ordinary managerial cadres to wrap their mind around of what a web company is and learn the whole model of behavior expected from them. The few who manage to learn some basic technical disciplines and go up in ranks tend to overestimate the importance of their experience.
You meet such people a lot in a dotcom setting. It takes great effort to persuade such person to bother to put efforts to understand yet another mentally voluminous subject that will break his idea of "cool" yet another time.
It is like trying to persuade a prideful child who just learned how to drive a tricycle to learn to drive a normal bike...
No, I don't work for Microsoft. However, I have enough experience with their ecosystem to suggest that their revenue focus is not in web dev. Other products (such as Windows, Office, Azure and Xbox) are their prime source of revenue. Whilst I don't doubt they have plenty of web devs (TypeScript and VS Code both spring to mind as web-based tech from Microsoft), I wouldn't say that is their core competency, so...
> "a company whose topmost technical expertise is underhanded web programming"
... doesn't ring true. However, if you know people on the inside I'd be interested in knowing how the size of the web dev teams compares to other teams, such as the Xbox division.
> Yet, what took them at least 5 years, and extremely expensive to make reticule limit chips, was bested by a no-name Chinese fabless that blew TPUs out of the water on power/cost/performance ratio.
I'm not familiar with what company or product you're referring to. What is it?
Novumind, they showed their engineering samples few months ago. And yes, they got quite close to their initial promise of 3 teraflops per watt.
Cambricon - possibly a scam, though also claims performance on an equal level with moderately sound technical approach. They never released much info other than fancy CG videos.
I can't find much information about Novumind (NovuTensor is their product) nor Cambricon. NovuTensor seems to be only doing inference , and only a specific type of inference (judging from their hardware, probably only images), which is a lot simpler to speed up than the training that TPUs do.
I'd take the statement "Unknown Chinese company beating Google with fewer resources and less time" with a huge grain of salt.
NovuMind only does 3x3 convolutions. Literally a hard burnt ASIC. Also, the TPU's efficiency is close. (Btw both their "FLOPs/watt " numbers are useless, they don't account for utilization)
And Cambricon... well, I'll just leave it at that...
I don't think that relevant. For running NNs, this is pretty much all what you need from an asic hardware - raw floating point matrix multiplications per second. And measuring how much power it takes for it to do so is a good measure of power efficiency.
Surely, he is a top cadre, and an accomplished academician. The number of his students serving on CTO level jobs is in double digits.
The fact that Google went so far with cadres, and said to be throwing high six digit salaries even on people who came to the unit fresh out of universities clearly signifies the extend of their efforts and commitment. Their intent seem to be to bang money on it until it works, no matter the cost.
>Hardly clueless and inexperienced...
I never challenged the technical expertise per se. I'm saying that a generic web/dotcom/clickfarm business can't be normally turned into an engineering company of an inherently different nature, regardless of how much money will be banged on the exercise.
People say that for the past 5 years, the TPU unit was running effectively, like a research institute of some kind: regular workshops with academicians, amount of research works written exceeding the amount of code, and so on.
TPU unit people did their job splendidly, but Google's managerial unit that authorized the whole affair seem to me to be having hard time wrapping their mind around the question on how and what to do with it.
On the other hand, Ren Wu, being originally a semi engineer, had a very clear idea what he wants from the very start: off-the-shelf i/o ip, axi bus, wide registers, sram fifo, directly register fed matrix multiplication units, and predominantly synchronous operation. Voila. No talkshops, company being turned into research institute, or six digit salary cadres what so ever. The chip might well be a one man project.
To clarify, Dave doesn't head the hardware efforts (I doubt he wants the management headache!). Norm Jouppi was the lead of the TPU efforts. But Dave absolutely is part of it, he just doesn't have to herd the cats. :p
To the previous parent, the TPU didn't take five years - I don't know where you got that. But, in any event, I'd argue the key innovation in TPUv1 wasn't actually the design of a specific optimized processor: It was seeing the need for a year or two before anyone else in industry did!
Also - I think translation problem, what's "reticule limit"? In any event, if you're comparing what someone produced today with TPUv1, that's a bit of a weird comparison considering that the v2 is live and available in beta. :)
But, that said: For training, it's odd that you'd criticize the die area of the TPU when the major competitor is Volta -- 815mm^2 (!). It's pretty clear that TPUv2 and Volta both have similar aims in terms of high memory bandwidth (both use HBM), monstrous matrix multiply throughput, and a high-speed interconnect to be able to scale to larger clusters.
Making a single-chip inference engine, which NovuTensor appears to be, is a very different thing. So it's better compared to TPUv1, which is also an inference engine. I can't find the die area of TPUv1 out there, but it's not a monster, as you can probably infer from its 75W TDP.
It would be helpful in this discussion to be precise whether you're comparing to TPUv1, Cloud TPU (the v2), or something else, and if you're talking about inference or training.
(I assume my disclaimer is already obvious - I work part time at Google Brain - and that everything I'm writing is my own opinion, etc., etc., etc.)
This thread is full of well-written arguments that don't offend anyone. I understand that these points may be wrong, but these comments sure as hell shouldn't be downvoted.
Apple is company with an own silicon, but it will never be a true microelectronics company.
Google's TPUs are another examplary case. Goog did spare no money to get into the market first and secure a tech/platform lockdown. Yet, what took them at least 5 years, and extremely expensive to make reticule limit chips, was bested by a no-name Chinese fabless that blew TPUs out of the water on power/cost/performance ratio.
All American dotcoms that are actively trying to get into hardware are oblivious to the fact that there is an inherent difference in between a hardcore engineering company to a company whose topmost technical expertise is underhanded web programming.