Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Counter-argument: Just let it sit there and keep working? Outside of security patches, if there even are any, there's always the option of "just stop touching it." After all, the whole pitch of elastic compute is you only pay for what you use. It's not like resources are being freed up for other things here.

The software industry is always on this treadmill of churn, leading to stable products continuing to have a constant maintenance burden. But that's not an intrinsic property of the area. It's a thing we do because... we can? We want to? But it's not required. Silo it off and just leave it be.



> Counter-argument: Just let it sit there and keep working? Outside of security patches, if there even are any, there's always the option of "just stop touching it."

That's not exactly possible with a monorepo that has no branches and everything has to be maintained 'in step'. Any time a library breaks API, all dependents must pass tests, and/or be fixed so that they do. Any time a runtime API breaks (think things like interfaces to runtime authentication, compute scheduling, database services, network bandwidth scheduling), updated services must be made to conform and then be rolled out. As with real life, this might mean getting a 1 or 2 year old deployment system (configuration for production, roll out code, ...) undusted, understood, probably fixed in turn, etc.

As such, some SRE/SWE team _must_ be responsible for projects that are in the "don't touch"/maintenance phase. This is very unsexy, toil-y and you don't get promoted for it - and as such barely anyone wants to do it at Google.


> As such, some SRE/SWE team _must_ be responsible for projects that are in the "don't touch"/maintenance phase. This is very unsexy, toil-y and you don't get promoted for it - and as such barely anyone wants to do it at Google.

That's Google's problem, and it's a problem they might want to look into solving if they want to turn around their reputation for killing products.


Google pays their engineers a lot and they’d need to recruit a lot more of them if they were going to keep running everything they ever killed. That, or stop launching so many new products.


I know it's beside the point, but you stepped on one of my peeves. Google doesn't have to pay their engineers "a lot". They don't have to keep sucking more and more wealthy people into a concentrated corner of California. They could open a modest office in Nebraska and pay some 5-figure salaries.

They could hire some run-of-the-mill college grads to work on a "dead" project. They could cut their teeth on low-profile stuff within Google and perhaps demonstrate abilities that move them up the ladder.

I'm not saying that Google should do this to keep everything they've ever touched alive. But spreading the talent around geographically would solve a lot of problems for Google and for California and for the world. Same for all FAANGs.

Quit earmarking $billions to solve the housing crisis in SF. Spend $1M in St. Louis and move 100 employees out of SF. Then spend $1M in Amarillo, and $1M in Sioux Falls. Shucks, maybe splurge and spend $20M here and there. $250M to build up 10 offices around the continent will do more to solve the housing crisis in CA than $1B spent in CA. And it would cut FAANG costs in half at the same time.


> Spend $1M in St. Louis and move 100 employees out of SF.

~100/100 SF Google engineers would change teams if they received a mandate that their team was relocating to St. Louis (or basically anywhere). Anyone who couldn't arrange to change teams would quit.


The fact that Google has 70 locations around the world, including 26 around the US, indicates that some Google employees are happy to work outside of SF.


Yes, but not a significant number of the ones living in SF. The parent comment was suggesting to "move 100 employees out of SF", which is clearly not happening. There are 70 locations around the world that these people could theoretically work and they have chosen SF. I'm sure pretty much every person working at each of those 70 locations was hired to work at that location. I'm am a bit appalled by the shittiness of the viewpoint expressed by the original comment, that the solution to lots of people wanting to live and work in high-demand areas is to force them out by moving their jobs somewhere they don't want to live. There are a lot of great reasons to want to live in a global hub city and work at a centralized office instead of working at a satellite office in Amarillo or Sioux Falls.


Recently Google has actually been low balling a lot of candidates and is nowhere near top of market. In many cases they are refusing to match competing offers, and are even offering people less money than what they are currently making


Heck, they just reached out to me, which means they’re scraping the bottom of the barrel.


Not true. I haven’t been contacted yet so they definitely haven’t reached the bottom. :)


Exactly what happened to me. I had offers from Google, Airbnb, Lyft and Uber. In terms of numbers, Lyft > Airbnb > Google > Uber.

Google tried to lowball me all the way through, and asked me to share screenshots of actual emails from other companies to prove my other numbers. They increased the numbers but frankly, they weren't anywhere close to Airbnb/Lyft. They did not act like a company that would beat other offers for talent. My offer was in the AI Assistant team under Google Search, so it wasn't an orphaned arbitrary product team either. Broke my heart coz I was so keen on joining Google from the start.

Hope this helps somebody out there!


The company is so massive now that it actually has an overabundance of engineers.

Small projects for maintenance don't need entire teams either and can be completely handled by 1 or 2 junior members.


Gee, then I hope they figure out a viable biz model to fund all that producting.


This. Someone has to be in OWNERS and it’s difficult to find people willing to do this because fixing breakage due to API changes isn’t going to be on anyone’s OKRs. And if it’s not on your OKRs, it’s not contributing to your promo.


Which proves the futility of these OKR metrics.

Once the metric becomes the objective, it loses any usefulness, indeed it becomes the opposite


How is that true? The OKR explicitly doesn't include cloud print because it is deemed not important to Google, and so cloud print is not worked on. Seems to be WAI?


Seems then like the incentive systems have a blind spot: they don't account for pissing off users relying on a service.

Why shouldn't people get promos for doing heroic acts on obscure services that actually delight users in real life? That surely has a benefit for Google. And the lack of it has a cost. Think of it like a PR budget. They'd get fewer grumpy articles about unmaintained services being dropped.


> This is very unsexy, toil-y and you don't get promoted for it - and as such barely anyone wants to do it at Google.

I don't think this is google specific. Keeping a dead product on life support is going to get boring for most people after a while.


It’s a feature not a product. Not everything has to be about profit and loss if it’s part of a larger ecosystem. Google Cloud Print was a feature of Android as well as other parts of Google’s ecosystem just like AirPrint is a feature of iOS/MacOS.

But Google has always been piss poor at managing an ecosystem. Just look at the state of Android updates.


There is an infinite supply of cheap intern labor at every college in the U.S. chomping at the bit to mop the floor at google hq.



Did you read your own link? Chomping at the bit has the same meaning.


"Still, if you’re writing for school or for readers who are versed in English, champing at the bit is probably the safer choice."


[flagged]


Neither productive, insightful, nor accurate, just FYI.


that’s a problem, yes, but orthogonal to it being in a monorepo. the google monorepo is NOTHING like a git monorepo


A lot of the responses to the parent are things along the lines of "well it needs to be owned by a team" and all the usual valid cost-related reasons why [x] can't be maintained, and of course that doing so is boring.

Isn't this the kind of project that should be reserved for people in explicit learning states such as (for example) novice hires, new hires, and aspiring mangers who haven't done any management before? It doesn't really have burning deadlines, it's not a frontline project but it's still something that can be referenced as work with impact, there's still the technical challenge of keeping it functional as the years go by, and eventually it will actually be old enough to be taken out back and deprecated. If any company could do something like this with their various projects as well as culture, I'm pretty sure Google could.

Of course there are probably a bunch of problems with this idea, but it seems to be more preferable in contrast to just axing a product that a not-insignificant number of people found useful and going "NEED. NEW. SHINY. TOY. MUST MAKE MONEY NOW!"


I think if they release it as OSS some people would eventually fork it and maintain it.


Except it doesnt work, don't forget that endpoints ( *.google.com or other google subdomains) are hardcoded in the printer firmware out there.


Over time, if the cost of creating a new thing for a company is maintaining it forever, a lot less new things will get made.


I'm frustrated at the fact that this comment, and many others elsewhere on this, are creating the false dichotomy that Google either needs to ruthlessly kill products that many users are still using (Reader, Cloud Print, ...) or alternately support Every Single Product Forever.

They can still kill their outright failures, of which there are many.


> Silo it off and just leave it be.

Not really possible. It's an internet service that talks to large number of different printers from different manufacturers.

Besides the code management issues others have mentioned, services like this require a great deal of product and partner coordination and testing effort to ensure that even small security patches don't break the heterogeneous ecosystem of devices.

More importantly, CUPS and driverless printing standards have solved the problem (os-specific print drivers) that cloud print was designed to work around.

Disclosure: Google employee but not on cloud print, though I use it on my personal printer.


> Not really possible. It's an internet service that talks to large number of different printers from different manufacturers.

Of course it's possible. Those printers are not getting updates, either. Nobody is changing the printer side of this. And if they were it'd be a change in the protocol. It's easy to just freeze the protocol (which already happened years ago), and if any printer manufacturer wants something new then can go fork off and do their own thing. That's very different from breaking something old which is what's happening here.

> More importantly, CUPS and driverless printing standards have solved the problem (os-specific print drivers) that cloud print was designed to work around.

CUPS still needs printer-specific drivers to do the actual printing part. And how widespread are driverless printing standards? Do such printers actually exist in consumer households? My Dell printer from just a few years ago certainly isn't.


> Those printers are not getting updates, either. Nobody is changing the printer side of this

Of course things change. Security protocols get upgraded. New attack vectors are discovered in protocols and need to be fixed. New devices with new capabilities are created and need API integration. That all requires human effort to make happen. All of that has to be worth the cost of maintaining the project.

> CUPS still needs printer-specific drivers to do the actual printing part.

Even if it needs them, the point is that it has them now, probably due in no small part ot the fact that CUPS is used on the Mac.


> Of course things change.

Except they don't. The clients are frozen in time forever. There's no change happening here. It doesn't matter if they should be changed, they aren't being changed. Printer firmware does not get updates.

> New attack vectors are discovered in protocols and need to be fixed.

IF that ever happens, which is super duper unlikely, then kill it since the clients are unfixable. But this is not an ongoing cost. There's no continuous change aspect to that at all.

> New devices with new capabilities are created and need API integration.

That's not change, that's new features. As I said, simply tell those things to go do their own thing. That's unrelated to leaving Cloud Print on life support.


> Printer firmware does not get updates.

Sounds like you've never owned a printer before...

> IF that ever happens, which is super duper unlikely, then kill it since the clients are unfixable.

So have a possible security hole with no active developers until a third party finds a vulnerability, then shut it down?

> But this is not an ongoing cost.

You would still require SRE support for keeping this thing up running. You would also open yourself up to a lot of risk, losing user trust and potential lawsuits if it ever was hacked.

Everything you wrote sounds like an elaborate troll but I'm assuming that you've just never worked on a large system before.


https://www.google.com/cloudprint/learn/printers/

Yep, to highlight some problems on the printer firmware side:

- outdated RootCA list, forcing Googles endpoint to stay on old CAs

- devices not supporting newer TLS versions like TLS1.2, TLS1.3 etc

- ossification of auth mechanism


> So have a possible security hole with no active developers until a third party finds a vulnerability, then shut it down?

This is true of every internet protocol. Being staffed at all does not mean the protocol or server has active security research being done on it. It usually doesn't. Taking people off of the work of constant churn from refractors and internal tail chasing doesn't actually change much.

> Sounds like you've never owned a printer before...

Sounds like you're just trolling but I'm going to assume you've just never seen a printer or worked on a stable system before.


And how widespread are driverless printing standards?

Very widespread. Most consumers printers support AirPrint for iOS and MacOS.


What are you going to use after this? A internet available print server? VPN?


The problem I had that cloudprint solved was printing from a Linux machine to a local non-Linux-compatible printer. With CUPS and driverless printers, this is no longer a problem. I don't generally need to print remotely to my printer.


Well... I do and I choose my printers based on availability for remote printing.

You forget that there are a lot of cases where elderly people can't print shit and need actual paperwork.


> You forget that there are a lot of cases where elderly people can't print shit and need actual paperwork

I'm guessing you are referring to a scenario where you have configured an elderly person's printer to allow you to print directly to it, sort of like a one way fax machine? I can see how that could be useful for some people, but it's a very rare use case, and hard to argue for maintaining the cloud print service based on that.

There appear to be alternatives for this specific use case, like HP ePrint, or you could set up a VPN to use in these situations. You could also just use printer that hooks up directly to eFax: https://www.faxcompare.com/blog/signing-up-for-efax-with-you...


More importantly, CUPS and driverless printing standards have solved the problem (os-specific print drivers) that cloud print was designed to work around.

Guess who owns CUPS?


No one? It is an open standard originally created by Apple.


Apple owns the copyright to the CUPS software that runs on Linux and Macs.

But the point being that unlike Google, Apple has supported CUPS and actively developed the software for well over a decade even though it doesn’t profit directly from it.


Having customer data is a liability. If someone hacks this service and gets the ability to read every customer's printout, it would be a disaster for Google's reputation. If it doesn't make enough money directly or indirectly to compensate for the liability it presents, it is actually a VERY expensive service to continue operating.


> Just let it sit there and keep working?

Sounds simple but sometimes it is not so easy.

E.g. it might be written in Python 2 for all we know. Should they leave it on Python 2 with zero security updates, or invest lots of time porting to python 3?

Obviously there will likely be a million other internal examples of the python 2 to 3 migration that we don't know about.


> Should they leave it on Python 2 with zero security updates, or invest lots of time porting to python 3?

Leave it, of course. The only meaningful security risk at this point is something like heartbleed. We're talking security issue in 10 year battle hardened protocol. This is a fantastically rare case. Not something you need headcount on a constant basis to deal with.


"250M printers compromised by Google Cloud Print" is an ugly look. It won't matter at that time that Google rescued a beloved product from being Deep-Sixed.


Right. Is nobody remembering the Google Plus API fiasco?


So this works in theory until there is a really bad bug in some part of the code base and there is no one who knows how to fix it. Or it breaks some kind of migration plan. Or it is embedded somewhere on some architecture that slows down an important projects development. or makes an application incompatible with a future OS

If they’re not going to keep a team dedicated to keeping it alive, then you get into a situation where an emergency fix needs to be deployed or it needs to be abruptly cancelled

It’s possible this already happened once or twice which is why they had to sunset it


> So this works in theory until there is a really bad bug in some part of the code base and there is no one who knows how to fix it.

10 years of production usage tends to have hammered out all the "really bad bugs." Sure it's possible that something goes wrong, but it's also extremely unlikely that it does unless something in it changes. Which, if you freeze it & silo it off, doesn't really happen.

A huge amount of software not made in silicon valley operates exactly like this. It just sits there, running for decades, doing its thing.


I don't know. I'd bet anything that has been production for 10 years with no feature added means anybody who knows anything about it left ages ago.

Its a cloud service that was started pre-Google Cloud. There is no way that it can sit in perpetuity and not be worried about old versions of Java, old OS, HTTPS bugs, large deprecation plans


Still, the risk and maintenance/ownership burden is there.


Yes, of course, but the point is it's a very, very small cost. It's basically zero for a company with Google's resources. Meanwhile killing it is actually costly. Not to engineering, but to PR and reputation.


Microsoft still patched products so old they had lost the source code by editing the binary itself!


I've done this before. It's really fun!


> Outside of security patches, if there even are any

On the fundamentals, a service the whole point of which is to let untrusted computers from the internet connect to your home network and talk to your devices is pretty likely to need continuous security updates.


It's not free to run and it is not generating revenues.

Have printer manufacturers pay for it's maintenance, then it's a reasonable argument for jut keeping it up.


> It's not free to run and it is not generating revenues.

If there's only a few users then it's cheap to run. If there's a lot of users then this is a mountain of bad PR right at the time they are trying to get users to adopt new products (like, say, Stadia). Since all it does is shuffle PDFs & images between endpoints I'm gonna guess this is on the order of basically free to run.

But either way canceling it isn't free, either. I'm arguing the cost of canceling this is far larger than the cost of keeping it running.


I saw it as a counterpoint to AirPrint. I'd like to know what changed that made Google think they don't really need this anymore. Maybe they think WiFi Direct is going to be the future?


Local wifi-based printing and printer discovery (same as Airprint) is already available on Google devices. No Cloudprint required.


But you still need individual print drivers.


Not for CUPS compatible printers.

Also HP provides its own apps and integration for local printing from Chrome: https://chrome.google.com/webstore/detail/hp-print-for-chrom...


Google is a large company. Surely the goodwill of keeping it around must mean something? Can’t they take a small loss on one of their “nice-to-have” products?


perhaps even charge people for it? like... it's part of a google business accounts?

unless it's going to bring in multiples of billions of $ per year - and accelerate that growth - they're probably not interested in doing that though.


You mean, have printer manufacturers pay to maintain software that allows users to continue using their old printers? I don't see that happening.


See: Google Play Music


Which I still prefer to YouTube music, it's theoretical replacement


Others have pointed out how this is hard with the Google code base (internal API changes, etc).

But it's more than that. Consider how something like GDPR impacts this. It requires at least some work from Legal, Product, Eng, etc. And these things happen fairly often. Even without a monorepo it takes work to stand still.


At Google nobody gets a promo for maintaining stale product.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: