Somewhat adjacent question: are there people working on ways to verify that a particular server or API backend are running the specific signed release that is open sourced? Can a company somehow cryptographically prove to its users that the running build is derived from the source unmodified?
[AD] Oh wow! I've not found someone working on the same issue, I wrote a short paper on a method to identify remotely running software [1]. I'd love to get your opinion!
[AD] I've wrote a short paper on this; Remote Software Identification -- Zero Trust Remote Software Verification [1] after reading a blog post by GUIX - and I've provide PoC code in Go [2]. It utilises a hash chain log server side, of inputs and outputs, every response from server embeds the related hash chain entry, client side can request and replay the log and verify the response hash adds up to the hash they calculated.
You can get most of the way there with something like the SLSA/BCID framework, with the final artifact including some trusted provenance from an attested builder. You could go further and aim for full reproducibility on top of the provenance, but reproducible builds across different environments can get messy fast if you're looking to independently build and achieve the same result. Either way the end result is you have some artifact that you reasonably trust to represent some specific source input (ignoring the potential for backdoored compiler or other malicious intermediate code generation step).
Now for the last mile, I'll admit I'm not particularly well-versed on the confidential compute side of things, so bridging the gap from trusted binary to trusted workload is something I can only speculate wildly on. Assuming you have a confidential compute environment that allows for workload attestation, I imagine that you could deploy this trusted binary and record the appropriate provenance information as part of the initial environment attestation report, then provide that to customers on demand (assuming they trust your attestation service).
Azure attestation service could be an example of a confidential service where you can find the binary measurement. It is a good example as you need to trust that service to attest your confidential workloads. The problem obvs is that it is not open source but they have some samples to deal with their measurements.
Conceptually, you nailed it with the last mile. You want to tie the runtime service's attestation report to the provenance that was generated at build time (and signed by the attested build environment). You can do this by including a copy of it with the service and serving it directly; or the build environment can publish it to a place where clients can later look it up using the Confidential Computing service's attestation measurements.
Yes Intel SGX gives you the enclave measurement which you could reproduce (if source code code is reproducible), the measurement can be verified against Intel keys to prove it came from that hardware. Similarly AMD SEV-SNP gives you that, it is preferred to SGX due to the ability to run VMs as opposed to smaller applications.
AWS has their firmware image in OSS to be able to reproduce it and then compare the measurements in their Nitro instances: https://github.com/aws/uefi
Azure has confidential compute offerings as well, their attestation, mhsm, ledger services rely on it.
But it is easy to talk about confidential compute and the link between the measurement and the source code. Such link does not exist in regular non-encrypted services and you basically need to trust the service provider about the proofs they give you.
Detecting physical ingress in a co-location server is not uncommon after contacting political representatives in some countries. It is wise to have password protected SSL certs as the bare minimum non-resettable tripwire, close monitoring of the HDD/SSD drives s.m.a.r.t. firmware power-cycle counter, and of course an encrypted partition for logs and other mutable/sensitive content. Note for performance, a "sudo debsums -sac" command along with other tripwire software can audit unencrypted system binaries efficiently. Most modern ephemeral malware (on android especially) is not written to disk to avoid forensic audits assigning accountability, as the chance of re-infection is higher if you hide the exploit methodology.
Folks should operate like they already have someone with a leaked instance of their key files. In general, a offline key-self-signing authority issuing client/peer certs is also important, as on rare occasion one can't trust 3rd parties not to re-issue certs for Google/Facebook/Github etc. to jack users.
Eventually one should localize your database design to specific users, and embed user action telemetry into a design. i.e. damage or hostile activity is inherently limited to a specific users content, sanity checking quota systems limit the damage they can cause, and windowed data-lifecycle limits the credentials to read-only or does garbage collection after some time.
In general, the rabbitMQ AMQP over SSL client signed cert credential system has proven rather reliable. Erlang/Elixir is far from perfect, but it can be made fairly robust with firewall rules.
