Also, have you considered using i3 instances for their superior price per throughput/IOP? Obviously, you can't rely on EBS snapshot based backup/restore workflows on those instances, but maybe the performance gain is compelling enough to make the exercise worthwhile?
I have rejected this on the basis that stopping and starting an instance is too useful for solving problems. Even with a few striped EBS disks, instance issues solvable by re-attaching the disks via stop/start are probably ten to one hundred times more likely to occur than a disk failure.
Still, I know some people do things this way, relying on archives and HA for redundancy, and it can be pretty good.
As a random sample of operating time, probably latency constrained, where something like a local disk would be nice. We do have occasional spikes for bulk writes and reads, and the good throughput of large IOPS EBS are instrumental there.
Does Citus (Cloud?) have features that offer better high availability and failover functionality than what RDS provides? Managed Patroni and packaged workflows for zero-downtime failover would be quite interesting, but I don't see anything like that mentioned on https://www.citusdata.com/product/cloud.
We don't use Patroni or any of the other off the shelf items. We rolled our own primarily from our years of experience on Heroku Postgres. We're actually working on a detailed post on how HA and disaster recovery works for Citus Cloud, though the core mechanism powering it all under the covers is our state machine. You can read a bit about how it works in this post: https://www.citusdata.com/blog/2016/08/12/state-machines-to-...
"In addition to continuous protection which is explained above, high availability is available if your application requires less exposure to downtime. We provision stand-bys if you select high availability at provisioning time. This can be for your primary node, or for your distributed nodes."
https://docs.citusdata.com/en/v6.1/cloud/features.html
Please let me know if I've missed any resources on this topic!
Yep, our docs are definitely behind the product here, we'll be working to improve. FWIW it would be the closest to the Heroku Postgres HA as it's the same person that wrote both systems.
What's the difference between downloading and executing a binary, installing a package (apt-get, pip, gem, etc), and curl | sh which makes the later so bad?
A package is downloaded completely from the net, then checked for it's signature.
A network transmission is by orders of magnitude less safe with regards to corruption of the payload.
If the script source is on github and isn't run under sudo, is there a meaningful difference between curl | sh and apt-get install from a PPA, gem/pip install, etc?
Meaningful? In most cases no, but since we're already talking about security, curl'ing the shell script from github exposes you to another attack vector, like MITM'ing the script.
The previous generation had a small instance (m1.small). There is no comparable offering in the current generation (at least, not yet). The smallest m3 instance is the m3.medium. You can still provision an m1.small, though, right? You just miss out on the hardware upgrade.
GCE, Digital Ocean, etc may be a better fit than AWS for these kinds of instances.
Previously a 1TB, 4K PIOPS volume was $525/mo. With a 35% discount on PIOPS this now runs $385/mo, but you can also get 1TB, 3K PIOPS General Purpose volume for $100/mo. Pretty nice price drop!
Generally, since 1 GB of GP SSD costs as much as 1 PIOP, in most cases you should just purchase a max(DESIRED_VOLSIZE, DESIRED_PIOPS/3)GB GP SSD volume rather than a PIOPS volume. I think.
The 3,000 IOPS figure for General Purpose SSDs comes with two caveats - it is available only in bursts for up to 30 minutes and it comes out of a capped reserve which is slowly refilled over time based on the size of the drive. In other words, it is not Provisioned IOPS.
If you wanted it to be true PIOPS, the cost of a 1TB 3K PIOPS SSD is $425 (1,000GB * $0.125 for storage plus 3,000 PIOPS * $0.1 for operations). (EDIT: this figures are wrong; please see responses below for the right math).
EDIT: My understanding was incomplete - General Purpose SSDs can burst up to 3k IOPS, but they also provide provisioned IOPS at a rate of 3 IOPS per GB. Effectively, 1TB drive then does provide 3k PIOPS (3 * 1,000GB) and bursting limits are only a factor for smaller drives.
Thank you for correcting me. I disagreed after reading the same blog post, but after your explanation I realised I completely missed the baseline performance guarantee. Your math was spot on!
But note that the baseline performance for the new General Purpose SSD is listed as the size (in GB) times 3 IOPS (so 3K/3072 IOPS for your 1000GB/1TB example). So large volumes will have a baseline that basically matches the listed "burst" IOPS speed, and at less than 1/2 the cost of the PIOPS version.
It looks like 3 IOPS per GB guaranteed, even when throttled, and much higher when bursting. That's still 3000 IOPS (3 X 1000) worst case for a 1TB volume.
I agree with the gist of the post, but there are some ways in which you can tip the scales in your favor:
- Get an offer where you get your options up front and the company reserves the right to repurchase them.
- Exercise your options immediately and file an 83b election. There will be no difference between fair market value and strike price, so your exercise-time tax liability should be zero. Obviously, this is easier if #options*strike_price is low.
- Look for a low strike price. If the seed round was convertible debt, the strike price may still be low.
- If you're trying to "go big or go home", look for a big gap between strike price and the price of preferred shares in the most recent round.
- Work at a company that is a qualified small business at the time you (fully, see above) exercise your initial grant, and hold your stock for five years. This can make any capital gains almost entirely tax free (see e.g. http://www.morganlewis.com/pubs/Tax_LF_CongressExtendsSmallB...)
I am _not_ a tax/finance professional, so remember to take these with a grain of salt!
