I did something similar. I build a tool[1] to import the Project Arctic Shift dumps[2] of reddit into sqlite. It was mostly an exercise to experiment with Rust and SQLite (HN's two favorite topics). If you don't build a FTS5 index and import without WAL (--unsafe-mode), import of every reddit comment and submission takes a bit over 24 hours and produces a ~10TB DB.
SQLite offers a lot of cool json features that would let you store the raw json and operate on that, but I eschewed them in favor of parsing only once at load time. THat also lets me normalize the data a bit.
I find that building the DB is pretty "fast", but queries run much faster if I immediately vacuum the DB after building it. The vacuum operation is actually slower than the original import, taking a few days to finish.
I haven't tested that, so I'm not sure if it would work. The import only inserts rows, it doesn't delete, so I don't think that is the cause of fragmentation. I suspect this line in the vacuum docs:
> The VACUUM command may change the ROWIDs of entries in any tables that do not have an explicit INTEGER PRIMARY KEY.
means SQLite does something to organize by rowid and that this is doing most of the work.
Reddit post/comment IDs are 1:1 with integers, though expressed in a different base that is more friendly to URLs. I map decoded post/comment IDs to INTEGER PRIMARY KEYs on their respective tables. I suspect the vacuum operation sorts the tables by their reddit post ID and something about this sorting improves tables scans, which in turn helps building indices quickly after standing up the DB.
Holy cow, I didn't know getting reddit was that straightforward. I am building public readonly-SQL+vector databases optimized for exploring high-quality public commons with Claude Code (https://exopriors.com/scry), I so cannot wait until some funding source comes in and I can upgrade to a $1500/month Hetzner server and pay the ~$1k to embed all that.
I'm curious if you have tried SeaORM? I've used it a little bit (not too extensively) and really like it. It's like sqlalchemy in that you can declare your tables and have a type checked query builder, which is a big win IMO. It's nice to add/change a field and have the compiler tell you everywhere you need to fix things.
I've definitely had issues when using sqlalchemy where some REST API type returns an ORM object that ends up performing many queries to pull in a bunch of unnecessary data. I think this is harder to do accidentally with SeaORM because the Rust type system makes hiding queries and connections harder.
Most of my usage of SeaORM has been as a type query builder, which is really what I want from an ORM. I don't want to have to deal with lining my "?" or "$1" binds or manually manipulate strings to build a query. IMO a good query builder moves the experience closer to writing actual SQL, without a query builder I find myself writing "scripts" to write SQL.
This is what a number of startups, such as Yurts.ai and Vannevar Labs, are racing to build for organizations. I wouldn't be surprised if, in 5-10 years, most large corps and government agencies had these sort of LLM/RAGs over their internal documents.
Here is straw man proposal, similar to cert chains and webs of trust: Say I'm a "curator". I say on HN/Reddit/Discord "here is my key hash 'p2pcuration:185da2bc59167692f596404fd83235f9bcb4e107b041f2e6e8d972da6dba00b7'". Any user that clicks the link or copies it into the search app adds the key to the trusted user list. With my private key I can sign torrents after I download them myself, which would mark the torrent as "good". When anyone who has added my key searches, the system searches for a corresponding signature from me as well. If a signature is found, the UI can chose to elevate that result.
The system could be extended so that signers could also sign other keys, expanding the trust network.
This system doesn't need to be run or maintained by each user. It could be served through a webui that can be run locally or shared with a small community. Migrating the interface to a new host would just require moving the config and keys.
What happens when you do this is that a few or one people in the system become trusted above the others, network effects do their thing, and the whole thing becomes informally centralized around them.
I've met Ross during my time at Princeton and he is a really genuine person, he is not trying to ruin anyone's life. This incident is the result of an uncharacteristic blind spot in empathy: a mistake.
I also have experience with the Princeton IRB on similar topics. The reality is that Princeton's IRB, and IRBs in general, are not equipped to deal with this sort of online research. IRBs were created as a reaction to unethical medical research, in particular the Tuskegee Syphilis Study [1]. My experience has been that the IRB has a greater expertise on medical and sociological studies. This leads the IRB to having a very narrow view of its remit in other domains. Unless humans are in a very literal way "subjects" of the study, then the IRB doesn't see it as human subjects research. In this case the IRB likely saw "Free Radical" and other websites as the subject. In both my studies and those done by my peers, the responses on what is and isn't human subjects research is uneven and you will often get a generic "this study does not constitute human subjects research" response from the IRB. This can be the case even if there possible negative repercussions to the "not subjects" in your research.
For example, say your study involves testing the vulnerability disclosure policies. How well do websites respond to vuln reports? In your study you send out 100 vulnerability disclosures. After you report these vulnerabilities, a human may read your vulnerability report and make a decision based on it. This presents a risk that the individual security team employees involved in your study will be scapegoated and fired when you publish your (potentially damning) results. How do you balance the value this study provides the public against the risk to the individual employees' livelihoods? The IRB isn't going to help you do this balancing, they will just say "this isn't human subjects research".
IRBs quite simply aren't equipped to evaluate this sort of research at the moment. This can be frustrating for a young twenty-something researcher just out of college trying to do the right thing while generating impactful research. You come in thinking that the IRB will be a guiding hand of wisdom and prudence, but you are quickly disabused of that notion after most of your interactions feel like conversations with lawyers in a compliance department. Many researchers in "CS" don't even involve the IRB, because they don't always see the ethical dimension of their work, but the fact that Ross did shows that he was trying to do the right thing here.
I don’t doubt that Ross is a nice person, and I think he meant well. FWIW, I think this is a great thing to study and in other circumstances I’d be glad he’s doing it. But much as Ross didn’t intend for me to be hyperventilating, heart pounding as I imagine trying to explain to my wife how my little hobby is getting us sued, that’s exactly what happened. That was a whole awful lot of extra stress that I didn’t need.
I definitely see a problem in that some people think that if the IRB doesn't object to what they're doing, it's OK. But ethics is a responsibility of the entire research team, and the research team is usually far better placed to understand the implications of their research strategy than the IRB.
The following are big problems here:
- lack of informed consent
- deception
Researchers should be trained that those are only allowed in exceptional cases where the benefits outweigh the harms.
Well if you have informed consent, it's not going to be a problem. If you don't, then you need to do a more careful analysis of ill effects might ensue when someone gets the letter (feel distress, spend money on a lawyer).
There is often a tendency to dehumanize things when it involves sending stuff to corporations. Even in that footnote, it's not employers processing fictious resumes, it's people.
So it's much more likely you'd get approval to do something "to a corporation" even though 99% of the time, it's really still being done to humans
I'd like to disagree as someone who knew Ross during my time at Berkeley. He absolutely is intelligent and thoughtful enough to know what he was doing -- including the consequences.
Berkeley's IRB is similarly illed -- resulting, a lot of trust (i.e. empathy) is placed that the lead will not do anything as obviously unethical as this. This is not the mistake that someone as intelligent as Ross makes, this was a conscious decision that backfired.
The fact that Ross didn't mean to do this is all the more reason why someone - maybe an IRB, maybe not (your argument makes sense) - should be assisting 20-something researchers with having a well-informed perspective.
In the absence of an organization that's good at this (which doesn't seem to exist and should), this probably should be the supervising professors.
As a research group leader, I find it unfortunate that the grad student seems to be the public face of this and is therefore attracting most of the ire. Feels like the student is being thrown under the bus, and responsibility for ensuring the study is conducted ethically should ultimately be that of the principal investigator.
I hope this fellow Ross does not become suicidal or otherwise depressed when he sees the weight of the internet coming down on him for this faux pas. Ross, none of this wil matter in a year. Or 5 years.
Sorry, but this sort of attack is not ok on HN and I've banned the account. It's entirely possible to post substantive critique without stooping to this—as many HN users have demonstrated in this very thread.
Fortunately there are a lot of other more-senior leaders in larger companies and longer-lasting non-profits who believe people can learn from their mistakes and there is no magical class of people who never make mistakes. Obviously mistakes were made here, and obviously at least the PIs should have known better - but deciding that you're going to keep the student on some mental blacklist forever, and instead find people who haven't yet had the chance to learn from their mistakes, is short-sighted.
You've probably heard the story of IBM's Thomas Watson being asked if he was going to fire someone who made a mistake that cost the company $600,000 in lost sales. No, he said - I just spent $600,000 training you! Why would I want someone else to benefit from that training?
(Also, the fact that you are aware there might be enduring consequences to this comment if you associated it with your name, and are therefore keeping your name off of it, is ... interesting. My advice to Ross Texiera is that, until proven otherwise, the commenter above is some random troll in high school who doesn't know how the real world works. If they wanted the threat to be taken seriously, they'd post it on LinkedIn.)
I think it’s kinda funny how sending vaguely threading emails (suggesting violation of statutes) sailed right through an IRB, but Scott Alexander got the third degree for giving patients a survey. Description:
>> When we got patients, I would give them the bipolar screening exam and record the results. Then Dr. W. would conduct a full clinical interview and formally assess them. We’d compare notes and see how often the screening test results matched Dr. W’s expert diagnosis. We usually got about twenty new patients a week; if half of them were willing and able to join our study, we should be able to gather about a hundred data points over the next three months.
"Any systematic investigation (including pilot studies, program evaluations, qualitative research), that is designed to develop or contribute to generalizable (scholarly) knowledge, and which uses living humans or identifiable private information about living humans qualifies as human subjects research. See Definition of Human Subjects Research for more information."
"Research is as a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.
...
Examples of systematic investigations include:
Surveys and questionnaires
"
So far, we got it in one.
I'll skip the part of whether it's generalizable - it's clearly intended to be here.
"A human subject means a living individual about whom an investigator (whether professional or student) conducting research:
Obtains information or biospecimens through intervention or interaction with the individual, and uses, studies, or analyzes the information or biospecimens; or
<The or is about getting PII in more cases, but this study is not getting PII>
...
Interaction includes communication or interpersonal contact between investigator and subject.
...
"
Well, there we go.
Seems a lot more straightforward in various IRBs than you seem to say.
As an aside, lots of IRB's also have mass email policies and are required to approve the text.
Now, maybe Princeton's IRB does not have as clear a definition. I can buy it, in fact!
But honestly, it doesn't seem that hard. If you are going to simulate fake emails to humans, for the purpose of gathering their responses, you are in fact, doing human subject research.
It also doesn't seem very hard to draw bright lines:
1. If you are interacting with people to see what their response is, even by email, they need to consent.
2. Do not deliberately deceive humans.
(You can even modify #2 to "do not deliberately deceive humans without an IRB explicitly understanding and weighing the cost/benefit" if you like, but most of the time, you actually do not need to deceive humans)
It's also really really hard to believe someone went to an IRB, and said "i'm going to survey people by sending them emails from fake people that seem mildly threatening, and seeing how they respond.", and an IRB was like "yeah, that seems okay, it's definitely not human subjects research".
It's up to the researchers to explain precisely what thy are doing in an accurate way. Saying you are surveying websites is totally inaccurate and confusing.
If a sociological researcher was like "whoa, i'm not emailing people asking for their family histories", that would be human subject research. Instead, i'm just "retrieving directed graph data from remote email addresses". I don't think that would go over very well.
Finally, as for not seeing the ethical dimension of their work, there is an easy fix for this (IMHO):
Make ethics classes required.
In fact, in lots of places, IRB's wont' review things if you haven't!
I think their point was IRBs say information isn't about an individual when the individual would say it is. Everything you quoted depends on the word about. And UCI's policy refers to US regulations. Those regulations contain surprisingly broad exemptions.[1]
People talk about emailing web sites any time they don't know if it's a person or a company in my experience. And ethics classes don't give everyone the same understanding of ethics.
Quick meta-comment; this is useful and informative information about the original link - please don’t downvote because you disagree with the point of view. If you disagree, please add a comment and make HN a great place for discourse!
This paper was published when there were big fights going on about whether the block size should be increased. Om camp argued for the status quo, which was causing high transaction fees and long backlogs. The other camp argued for a larger block size, which would reduce or eliminate the transaction backlog. Eliminating the backlog was generally considered a good thing, and the arguments were mostly around the block size, but this paper essentially came in and said "wait a minute, we might actually need the backlog once the block reward is 0."
Not to disagree, but to emphasize that the idea that the only way to increase bitcoin capacity is increasing the block size is in error. You didn't push that idea but others do.
Nearly every release of bitcoin over past 8 years has increased capacity or efficiency, often both.
I think this is depends on how large the network has scaled at the time the block reward dies. If BTC is doing 100x visa levels, each transaction fee can have a tiny fee and mining would still be viable.
That being said, I don't BTC has any intention of scaling to that degree (or at all really). The BTC devs seem much more concerned with building second layer products like lightning network. All these products actually reduce the number of on-chain transactions and by extension, miner revenue.
The problem is that lightning is basically federated Paypal. You have to trust your specific provider to some degree. If he stiffs you, you can get arbitration on-chain. There are almost zero barriers to entry so Paypal can be replaced by Paypal 2.0.
I don't believe for one second that when you increase the size of the audience that anyone will care which provider they use. In practice that means they will just use Visa, Mastercard and Paypal, maybe even by skipping lightning entirely.
"The average energy consumption for one single Bitcoin transaction in 2020 was 741 kilowatt-hours. This was significantly more compared to the cumulative 100,000 VISA transactions with only an energy consumption of 149 kilowatt-hours."
You can. But it is not a good metric. Assuming the difficulty adjustments ramp down you could still mine blocks with a couple of raspberry pis.
The energy consumption and capital investment in hardware is Bitcoins security model. As it would require you to put in the same amount of HW and energy to subvert the mining process. Probably we are at a point that this is almost impossible other than a state actor or a global conspiracy.
The block size increase/decrease is also tied to security. There are latency implications as well as the fact that some nodes might drop of if you increased the block size.
In both cases it comes back to security rather than the transactional throughput. I am a huge proponent but still kind of struggle with the idea of how much resources this thing sucks up.
But then again if it truly becomes the worlds ledger for wealth preservation... idk ... might be worth it.
Because that would imply linear scaling. There is a fixed energy cost that doesn't depend on the number of transactions. It goes up and down all the time.
The choice of the word “meaningfully” was deliberate . Of course you can do that. But I don’t think it is meaningful. The energy use isn’t a result of the transaction.
Feels like natural progression as underlying technology (internet speeds and HDD space) improves. So more network traffic can now get synced across more nodes faster.
One of the assumptions the authors make in this paper is that miners can turn their hardware on and off quickly, and that they will benefit financially for doing so. Mainly by paying lower electricity bills. It turns out the really big miners don't pay for electricity the same way you or I do. Big miners sign long term contracts for continuous consumption of energy, and don't save any money for turning mining hardware off for a short duration.
I know one of the authors personally and can try to forward on questions for anyone interested, though they have moved on from academic work so may have limited interest or context.
A much bigger assumption is that this will even be necessary, considering the last BTC mined will be in 2140. With such a long time horizon any prediction is basically fanciful guesswork; at that time miners might have all moved to renewables after they become cheap enough, we may have new forms of energy generation which make supply both super easy and cost negligible, or Earth may have entered a post-apocalyptic state and mining is no longer even a consideration. To assume no advancement in tech between now and then seems reductive.
The last BTC mined will be 2140, but the decay is exponential. The reward will halve in about 3 years, and it is likely to halve again 4 years after that, and so on.
The gold standard was the least of the Bretton Woods Era problems.
I also wouldn't call this new fiat regime with rising property prices and stagnant wages as a "net win" over Bretton-Woods, the gold standard was probably the only thing keeping the federal spending under control, and real wage growth up.
Price appreciation has to stop at some point. I doubt the price will rise to $10M after the 2028 halving so miners will be under substantial pressure by that time.
I read this comment with interest, thanks for sharing. Makes good sense to me.
If I may ask, do you think the distribution of bitcoin will be somewhat equitable as we approach 2140?
And do you you think it will actually become a medium of exchange as originally hoped? Or will it remain a store of wealth only (as things currently seem to indicate)?
And if it indeed remains only a store of wealth, will said wealth be distributed relatively equally, or will we be left with a relatively small pool of large holders?
Just trying to wrap my mind around the long-term evolution of bitcoin - and your way of explaining the energy conundrum above makes me think you probably have good insight on this.
Again I can't really say much about what could happen at this point. History "behaves" too bizarrely, especially over such a long time period, and inflection points are usually completely random and unpredictable (black swans), being only obvious in retrospect.
One thing I can be fairly confident won't happen is that BTC will become a medium of exchange. This is because BTC mechanically behaves more like a commodity than a currency, which means that (for various, immutable technical reasons) it's not able to match the speed of settlement, the price stability and the reliability of supply that are required for a good medium of exchange.
And I agree. I too doubt BTC will become the medium of exchange it was originally envisaged to be.
As to whether it will retain our confidence as a store of value remains to be seen.
I will continue to follow the heated discussion on HN with much interest. It's great to be able to read the consolidated thoughts of individuals who clearly understand this technology far better than I do.
It's impossible to predict the future. If everyone holds bitcoin as a store of value then the value should drop as it's not being used for anything one would think. I believe the best outcome is that people start using bitcoin for every day purchases and the hope is that by 2140 the price is relatively stable. Most likely there will be many large holders as there are with real money.
However, someone could maybe do a 51% attack on bitcoin at some point. Maybe there is a unforeseen flaw in bitcoin. Maybe something better comes a long. It's impossible to know.
" bitcoin for every day purchases and the hope is that by 2140 the price is relatively stable."
This is fantastically impossible.
It will about as stable as any other commodity plus the additional swings due to public stampedes one way or another, and the fact that there is no 'underlying value' to anchor the price (like Oil).
Feasibly, if exchange rates are razor thin (unlikely) then it might be possible buy milk and bread with BTC simply by doing the conversion on point, but that of course requires someone to do some 'investing math / intuition' every time they buy a stick of gum.
It can't be a functional currency, at least as it's designed.
And the notion that it consumes gigantic amounts of energy for no net value gained should have everyone up in arms.
> It can't be a functional currency, at least as it's designed.
Citation needed
> And the notion that it consumes gigantic amounts of energy for no net value gained should have everyone up in arms.
For what it's worth, bitcoin aims to deprecate every single brick-and-morter bank as well as all the armored trucks driving paper currency to and from them.
When you start to add up all of the energy that would be saved if bitcoin succeeds, the energy costs associated with mining start looking quite a bit smaller.
>For what it's worth, bitcoin aims to deprecate every single brick-and-morter bank as well as all the armored trucks driving paper currency to and from them.
Is this supposed to be some joke? No it doesn't. Someone had to create a fork of Bitcoin called Bitcoin Cash just to increase the block size. You've had your chance and you blew it.
What's going to happen from now on is that Bitcoin will be absorbed by Visa, Mastercard and Paypal who build an actual payment layer on top of Bitcoin and then people will use Visa, Mastercard and Paypal as their bank. It's the Bitcoin you know and love that is going to die.
The price of BTC has always been highly unstable. There is absolutely no evidence it will ever be stable, to the contrary in fact, all evidence points to the fact that it will remain unstable.
Every single commodity or currency extant to an economy is unstable relative to the currency of that economy. The only thing 'stable' in terms of USD are things closely tied to the managed USD.
Just the mere fact that it's highly unstable - all other things notwithstanding - make it impossible to be a currency.
"For what it's worth, bitcoin aims to deprecate every single brick-and-morter bank as well as all the armored trucks driving paper currency to and from them."
It's really bizarre that anyone would believe this.
Starting with the fact that 'digitization of currency' is already well under way and so there are no 'truck rolls' in the future.
But of course, the implication that banks only provide a 'place to store' money is completely false. Financial Services are an entire industry. Lending, transactions, authorizations, contracts, accounting, risk management, asset allocation, it's a gigantic industry.
But it's moot, BTC will be the same in 20 years that it is one, something to talk about.
Yeah, using the magic of DeFi I can easily get a loan for $100K with no bank approval and no wasteful bank employees checking my credit.
All I need is more than $100K worth of crypto assets I can tie up as collateral for my loan. So I'll be able to buy a car or a house, as long as I have more than enough money to do so already. Banks don't know about this one weird trick that makes them obsolete!
So Etherium will do credit checks? Evaluate the legitimacy and valuation of major assets? Validate small business plans? Check creditor references? Do due legal and IP due dilligence? Foreclose? Evaluate the credibility of executive teams? Evaluate contracts? Resolve commercial disputes? Do risk assessment of various kinds?
Etc, etc. etc?
The complete lack of understanding of anything financial postulated by the crypto crowd is really painful.
The notion of 'crypto' is actually interesting, and large swaths of the financial world would actually buy into it, but most of it makes no sense unfortunately.
Can't reply directly to the sibling comment on DeFi, but that solves the issue raised above e.g.: I have 100k worth of BTC, I want to spend that on something but I expect the value of my BTC to rise, so I use it as collateral for a pegged coin (say USDC or DAI) and make a purchase.
I am protected from losses of spending my BTC. As with all finance yes I also carry the risk of BTC falling and liquidating my loan - but assuming I have not also lost my loan I am protected from a major fall, as my collateral will be taken and I still have my loaned coins.
I dont know much about Ethereum but the mere existence of it threatens Bitcoin's "world domination" aspirations and who knows, maybe something will come out that will threaten Ethereum.
Or maybe there's a foreseen flaw in bitcoin.
It's based on public-key cryptography which can be broken by quantum computers. Those don't exist yet (at least not in any form relevant to cryptography in practise), but I think they will by 2140.
People often bring up the Bitcoin algorithm to make arguments against it, but don't seem to acknowledge the fact that the protocol is mutable.
If the sha-256 algorithm was cracked such that BTC blocks could be solved instantly, the existing miners would have to choose between:
1. No more income, or
2. Adopt a quantum-resistant protocol.
Market economics being what they are, I think it's safe to assume that BTC would survive the "quantum apocalypse." There's too much money at stake for any other choice to be the logical outcome.
From my understand, and I'm no expect, but the only known quantum attack against symmetrical crypto like sha-2 is [Grover's](https://en.wikipedia.org/wiki/Grover%27s_algorithm), and the recommended advice is to double the key size, so sha-256 would probably see a huge boost in "hash rate" but not be broken, a move to sha-512 would work probably work.
The problem is that Shor's algorithm breaks asymmetrical crypto used in the wallet signing, that means you can forge ownership of any transaction outputs, which would completely shatter confidence in the coin before they could migrate all ownership of all funds to a new post-quantum signature scheme, this problem is a lot harder to solve compared to a hash algorithm upgrade.
If they are physically realizable at all in practice, then they very likely will be by 2140.
But I think there's a non-negligible chance that the theory of quantum mechanics will break down as we move to superpositions of 2^1024 classical states that must be faithfully represented with physical elements.
The wealth distribution of bitcoin is even worse than the wealth distribution of society as a whole. Further, I suspect that this would be the typical state for any finite/limited asset. IMO, things like bitcoin (and gold) as a primary wealth store are how you create stratified societies with entrenched elites and banking orgs with more money/power than nation states. Not that bitcoin could ever happen dominate, but it's certainly not egalitarian in any sense.
I would expect it to get worse, over time, too. When the block reward was high, it meant that the (strictly financial) cost of actually operating the Bitcoin network was covered by Bitcoin dilution, which meant that everyone was paying it pro rata based on how much Bitcoin they actually own. As the block reward decreases, though, it increasingly has to be covered by transaction costs. That's going to progressively price people out of participation in the direct Bitcoin economy.
I can see using Bitcoin as a primary wealth store, similar to gold. I agree, it would have most the same social features, which seems problematic since none of that is really in line with Bitcoin's original political vision, but I'm not sure Bitcoin's original political vision is anything more than a piece of nostalgia cherished by people sitting on the periphery of the contemporary Bitcoin economy, anyway, so maybe that's no big deal.
The bigger problem I see with Bitcoin relative to something like gold is that it has some troubling practicalities. It's theoretically much easier to steal vast sums of Bitcoin in one go (because 1,000,000 BTC wouldn't be quite as subject to conservation of momentum as a tonne of gold bars would be), and it's definitely much easier for vast sums of Bitcoin to accidentally poof off into a crypotgraphic pocket universe where nobody can reach it anymore.
One that has a fixed block subsidy. After any amount of time, whether years or decades or centuries, it will have been distributed evenly over all that time.
Rather than having 50% distributed in just the first 4 years, and only crumbs in later decades.
I don't see how that logic follows, if this was a fixed block subsidy over the first 4 years instead of just 50%, surely the situation you are describing would be worse ?
There's a huge difference between fixed forever and fixed for only 4 years and then dropping to 0. Bitcoin is closer to the latter (instead of dropping to 0, it keeps halving every 4 years). Gold is closer to the former.
"Egalitarian currency" and "store of value" are somewhat mutually opposed goals.
A stable store of value is always going to socially/psychologically encourage hoarding and other human traits that lead exactly back to inequal distributions of value ("wealth").
A truly egalitarian currency would need to focus on the velocity and acceleration (the current of the currency) curves far more than the value at rest, which likely would be an afterthought.
>do you think the distribution of bitcoin will be somewhat equitable as we approach 2140
I don’t see why the distribution of anything would be “equitable”, outside of a communist utopia/dystopia. These days that word is mostly used by people exploiting the empathy of others to gain power for themselves.
I see what you're saying. Forgive my choice of wording. Perhaps "equitable" wasn't the correct choice.
I more meant whether it will become a currency of the people. Or a store of wealth for the average person.
I'm looking at this through an African lens. Bitcoin showed a lot of promise for cheap and reliable remittance payments and one day a fully fledged currency for the masses. Gave us Africans lots of hope in many ways.
These days I worry it is increasingly growing more difficult to acquire. It's no longer destined to benefit the people.
I don't mean to imply that everyone deserves an equal share. I hope I'm making sense. I guess I'm more wondering about its overall accessibility long term.
It's ironic. Bitcoin can only keep going up if it becomes more accessible and more people use it. Yet people resist making it more accessible. It's like they are betting on a failing technology stack that will blow up after x amount of users.
I don't see why Bitcoin's technology stack is failing if it's what is enabling its current value.
Keep in mind as well that the protocol is mutable as long as people agree to change it.
Saying BTC is doomed to fail as it's living alongside a multitude of other cryptocurrencies that attempt to solve its shortcomings is also a dubious proposition to me. Take for example the Mina Protocol[0], which eschews proof of work, huge blockchains, and slow transaction speeds for solutions built on top of zk-SNARKs.
If BTC has value as a long-term store of wealth, and if other cryptocurrencies exist to transact at faster rates with more liquid capital, and if that capital can flow between systems freely (as fast at BTC will allow), then every system is working as designed, right?
Exactly. I came here to say this, but you've said it quite eloquently.
I'll just add that some inequality is also a good thing. If you don't have differential outcomes then you don't have a meritocracy where "better" is rewarded. You have a system where everyone is equally poor like the USSR and there is no incentive to improve.
As a tangent, I wonder if Bitcoin mining could re-ignite the nuclear power industry. Nuclear power plants involve a high upfront capital cost, then produce a long-life of fixed electricity output at near-zero marginal cost.
My understanding is that one reason more plants were built in the 60s was because electricity prices were fixed by regulators. That made financial modeling easier, because investors could legibly forecast their payback on the capital investment. When electricity moved to free floating price markets, it introduced much more risk and therefore increased the cost of capital.
Seems like nuclear power and bitcoin mining are a match made in heaven. Demand remains smooth and predictable, even from minute to minute. The marginal cost of electricity is near zero. Many of the large mining pools are so large and well capitalized that they could afford to commission a dedicated plant.
Predicting nearly anything (today) about crypto over the timelines necessary to even only commission a new nuclear plant, let alone its lifetime, seems ... risky.
I wrote a bit of fiction recently which portrayed a future where a starship needed to run its own Bitcoin mining cluster (because "reasons") and the ship also happened to use a nuclear power plant. and for "reasons" the mining host hardware was also placed as close to the nuclear reactor as they could get away with (well the electricity generation portions anyway.) all needed for (arbitrary, fanciful) plot reasons, but one of ideas I used to justify it was because the power generation qualities fit the power needs of the crypto miners pretty well. stable load, low cost per watt, theres no wind or geothermal or tide in space, and solar is sometimes too weak and other times entirely blocked (picture situation when ship in planetary orbit, on far side from the star, so light is eclipsed.)
Most miners are on hydro, located right next to the dams, and probably colluding with them for either zero or near zero cost. Similar advantages to nuclear I'd imagine.
Yep. And I suspect there isn't any power cheaper than the output of an over provisioned wind or solar farm on a windy or sunny day. And they all will be over provisioned. In fact they all assume a faction of nameplate output now.
This is a good point that the authors don't touch on. There are many coins secured by sha256 and I'm sure many more will come along before BTC finishes it's release schedule.
Switching to any one of them is likely to provide more revenue than just attacking the network for a little extra transaction fee money.
This is one of those times it's really useful to make a distinction between bitcoin the currency and bitcoin the protocol. If it was the authors intention to describe a future problem on the BTC network then they messed up by not addressing the rest of the sha256 mining ecosystem. However, if it was their intention to describe a problem with the protocol itself then it kind of makes sense to not touch on miner's alternatives.
No, this is not what we are discussing. We are discussing using bitcoin's hashrate to mine other coins. No modification to bitcoin is needed, the other coins simply need to be aware of bitcoin and be able to verify bitcoin's blocks. Other coins can have an auxillary mining algorithm too, or even merged mined with other coins besides BTC, doesn't have to be sha256.
Miners do turn their hardware off quickly, we saw this in November 2018. After a long period of profit, suddenly only the top-of-the-line miners were making money, and they just threw out a pile of stuff. Writeup I did at the time: https://davidgerard.co.uk/blockchain/2018/11/27/the-bitcoin-...
Commercial energy prices do vary throughout the day though, so if they are signing a single fixed price contract per kWh that would miss the opportunity to run older hardware more profitably during cheaper times.
Besides, I doubt miners are keeping going during triad periods.
Absolutely! When miners are priced out they will be. If you can make a profit mining bitcoin, you’ll have already _arbitraged_ the pricing discrepancies in energy prices and made a boatload of money by now doing it.
And once you get priced out, you are operating at a loss. Then you turn the miners off, sell ‘em to the next person willing to gamble vs. doing their homework.
Re read that first paragraph real closely. I personally used to have a blog teaching people how to mine bitcoin on Ubuntu 10.04... until I didn’t anymore.
Now, whether anyone likes it or not: Bitcoin has both value and utility until at least 2040, block rewards incentivize mining for 20 more years (19, whoops) if we read the white paper.
Two thousand and forty. Every 10 mins, new block. Chain, new block. Huge billion dollar economy of SHA256 miners custom built for proof of work. That money to pay for all of that is coming from somewhere. Mark my words, proof of work doesn’t crash overnight. It wasn’t built built in a day either.
AFTER that, there’s 100 years of “will bitcoin live” because of minimal to NO block rewards.
These papers are so old and the subject matter is still misunderstood, it’s nice to see people learning and stuff but HN gah, just omagersh
the box on page 5 defines DefaultCompliant as "The default Bitcoin mining strategy, including all available transactions, mining on the end of the longest chain, choosing the older block in a tie, and publishing all blocks."
the core code does not choose the older block in a tie, the code chooses the block with the largest legitimate chainwork (under current TARGET epoch)
Second note: this paper investigates the era in the future when there are no mining rewards, only transaction fees. How far away is that ? How valuable is one satoshi now? Much of what is said is relative to that far-away case
(reading through the paper) This is an interesting (and thorough) thought experiment for what behavior might emerge when only transaction fees are the reward for mining. However the case the paper makes for a serious security problem gets weaker, as the argument depends on a growing number of assumptions as the paper goes on..
I fail to understand why a PETTY-COMPLIANT miner would ever realistically take a set of Tx that does not short-term maximize profit, given the competition for new blocks goes WAY up as the value increases. In other words, the undercutting and LAZY-FORK behavior would be crowded out right away, as it is insufficiently popular.
In the AGGRESSIVE-UNDERCUTTING discussion, it assumes a lot of "forks" or alternate chain tips, to chose from, is this true in practice? Are there really that many chain tips to chose from, to make this practice even a consideration?
I see in page 10 that there is a more refined and detailed descriptions on miner strategy. From a math perspective this is interesting, but the comments above stand.. would anyone even have a chance to move ONE BLOCK using these considerations, given the value and competition for every single block?
>It turns out the really big miners don't pay for electricity the same way you or I do. Big miners sign long term contracts for continuous consumption of energy, and don't save any money for turning mining hardware off for a short duration.
Really? Those contracts don't let them resell the unused electricity at (some fraction of) the spot price?
They talk about miner turning off the system just after they have mined a coin as they probability of getting another soon is very low while the cost of running the system same.
The paper that relies on turns out to have some inaccurate assumptions. The paper assumes that miners can flip their hardware on/off at essentially instantaneous intervals. The assume this to argue that miners will strategically turn their hardware off when the expected value of mining a block drops below the cost of power.
It turns out that many of the big miners have long term contracts with power companies to consumer power; they wouldn't save money by turning their hardware off for short periods of time. Power companies like this arrangement because it lets them predict demand better, and miners like it because they get "bulk rates" on electricity for being predictable in their consumption. The arrangement falls apart when miners turn their hardware on and off.
I am a PhD student in computer science at Princeton University. My current research explores the privacy of consumer genetic genealogy companies. You can read a preprint of my current research here: https://www.biorxiv.org/content/10.1101/531269v1
My work requires combination of engineering and simple (naive bayes) machine learning. For the paper above, I simulate populations with hundreds of thousands of individuals using Rust, and then analyze the simulated data and population using Python, with key functions optimized by Cython. The project required me to have a breadth of skill, from understanding bayes classification to CPU cache locality.
I am looking for either data science roles, engineering roles, or ideally roles that combine both. Having a research component is a plus! I will be available starting near the end of Februray, or the begining of March.
I did something similar. I build a tool[1] to import the Project Arctic Shift dumps[2] of reddit into sqlite. It was mostly an exercise to experiment with Rust and SQLite (HN's two favorite topics). If you don't build a FTS5 index and import without WAL (--unsafe-mode), import of every reddit comment and submission takes a bit over 24 hours and produces a ~10TB DB.
SQLite offers a lot of cool json features that would let you store the raw json and operate on that, but I eschewed them in favor of parsing only once at load time. THat also lets me normalize the data a bit.
I find that building the DB is pretty "fast", but queries run much faster if I immediately vacuum the DB after building it. The vacuum operation is actually slower than the original import, taking a few days to finish.
[1] https://github.com/Paul-E/Pushshift-Importer
[2] https://github.com/ArthurHeitmann/arctic_shift/blob/master/d...
reply