Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Hi. Cloudflare Workers tech lead here.

No, process isolation does not work for us, because it does not scale well enough.

The point of Cloudflare Workers is to distribute your code across Cloudflare's entire edge network -- 154 locations and growing -- so that you can respond to requests at the closest location to your end user. We want to put every customer's code in every location, rather than forcing you to choose a handful. Meanwhile, obviously, not every one of our locations is a mega-datacenter.

This means we need a way to support extremely large numbers of tenants per machine, with relatively low traffic per tenant (because each tenant's traffic is spread out over the world). We need this to be efficient.

This means, we need low memory overhead (to support many tenants per machine), very fast cold-start time (since cold starts are a lot more common in a decentralized scenario), and low overhead to switch between workers (because we're mixing traffic from all our customers everywhere).

Using embedded V8 isolates within a single process gives us 10x-1000x better performance on all these metrics than if we gave a whole process to each customer.

Relatedly, here's a talk I gave at Heavybit about the neat things you can do when you have fine-grained server compute: https://www.youtube.com/watch?v=YZSvJNBZsxg



V8's developer guide doesn't recommend running untrusted code in a shared process with "sensitive" data. How do you know if your different customers data is sensitive?

https://v8.dev/docs/untrusted-code-mitigations#sandbox-untru...


This is a relatively new recommendation on V8's part specifically in response to Spectre-type vulnerabilities.

We've spent a lot of time thinking about and building mitigations for speculative side channel attacks. For example, early on in the project -- before anyone even knew about Spectre -- we made the decision that `Date.now()` would not advance during code execution, only when waiting for I/O. So, a tight loop that calls `Date.now()` repeatedly will keep getting the same value returned. We did this to mitigate timing side channels -- again, even though we didn't know about Spectre yet at the time.

Chrome has indeed stated that they believe process isolation is the only mitigation that will work for them. However, this statement is rather specific to the browser environment. The DOM API is gigantic, and it contains many different sources of non-determinism, including several explicit timers as well as concurrent operations (e.g. layout, rendering, etc.). Side channel attacks are necessarily dependent on non-determinism; a fully-deterministic environment essentially by definition has no covert side channels. But, there's no way Chrome can get there.

The Cloudflare Workers environment is very different. The only kinds of I/O available to a worker are HTTP in, HTTP out, `Date.now()`, and `crypto.getRandomValues()`. Everything else is perfectly deterministic.

So, for us, the problem is much narrower. We need to make sure those four inputs cannot effectively be leveraged into a side channel attack. This is still by no means trivial, but unlike in the browser, it's feasible. `getRandomValues()` is not useful to an attacker because it is completely non-deterministic. `Date.now()` we've already locked down as mentioned above. HTTP in/out can potentially be leveraged to provide external timers -- but the network is extremely noisy. A practical attack would require a lot of time in order to average out the noise -- enough time that we can do a bunch of higher-level things to detect and disrupt possible attacks. It helps that Workers are stateless, so we can reset a worker at any time and move it around, which makes attacks harder.

NetSpectre demonstrated that even physical network separation does not necessarily protect you against Spectre attacks. There's simply no such thing as a system that's perfectly secure against Spectre, process isolation or not. All we can do -- aside from going full BSG and giving up on networks altogether -- is make attacks harder to the point of infeasibility. Luckily, we have lots of tools in our toolbox for making Spectre attacks infeasible in the case of Cloudflare Workers.


1. Thanks for your multiple well-informed, articulate comments.

2. "BSG"?


I think your parent is referring to Battlestar Galactica; it had no computer networks, and therefore was not able to be effectively attacked by the Cylons.


Yep. :)

The Battlestar Galactica was the only ship in the fleet that hadn't networked its computers, because the captain was paranoid. When the Cylons (AI) attacked, they instantly hacked all the other ships, but the Galactica stayed under human control and got away.

It's fiction, but I'm honestly really impressed with this bit of writing. It's both an entirely plausible and almost realistic strategy, and it gives the writers an excuse for the crew members to interact rather than let the computer do everything. Whereas on Star Trek one wonders why they bother with a bridge crew -- the captain might as well be punching everything into a computer directly rather than inefficiently giving orders to humans.


Do you see the complexity of V*'s multi-tiered JIT compiler as a security risk? Maybe a simple JS interpreter plus a simple AOT WebAssembly-to-x64 compiler would be safer.


Complexity is a risk, but we also care deeply about performance and startup time in much the same ways Chrome does. So we need that JIT.

I'm comforted by the fact that if there's a breakout in V8, Chrome is a much juicier target than we are -- and has an incredibly strong security team that will jump on the issue. I think we would be overall less secure using a lesser-known JS implementation.


Actually, I think Cloudflare is a juicier target, due to a combination of two things:

1. No process isolation 2. Data from lots of users on any given node

Am I missing something?


While I don't mean to deny the risk, which we take very seriously, I do think there's a number of reasons Chrome remains a much more interesting target:

* While process isolation makes Chrome somewhat less interesting than it used to be, keep in mind that process isolation is being rolled out as a defense to arbitrary-read attacks (like spectre), not arbitrary-code-execution attacks. Executed code can potentially talk to the rest of Chrome and perform actions that a web site is not intended to be allowed to. (At least in the past, this included the ability to read all the user's cookies, though it's possible they've improved on that, I don't know.)

* Cloudflare has potentially multiple tenants in a node, but the attacker has no control over who they are. There's no particularly good way to target a specific end user nor a specific web site.

* V8 vulnerabilities are likely to be memory safety issues, which are difficult to leverage into an attack without an understanding of the process's memory space, which is hard to get without having access to the source code or a compiled binary. The attacker would be flying blind and would almost certainly just segfault or be caught by our layer 2 sandbox (which blocks almost all syscalls), which incidentally would immediately alert us to their activity. (I don't particularly like this as a defense, since I like open source -- but frankly, it is a pretty big barrier.)

* The attacker would be burning their zero-day by uploading it to Cloudflare. We block eval() and the like, such that the only way to run code on our edge is by uploading it through our deployment APIs, which keep a copy.

* Last I checked, Chrome still does not use site isolation on mobile, because the overhead is too high.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: