Mining and proof-of-work
Last updated
Last updated
We now have a network of nodes distributed across the world that are all attempting to keep track of what bitcoin transactions have taken place. Although nodes are part of the same large network, it is not possible for them to directly communicate with all other 100.000s of nodes at the same time. They are instead communicating with only a dozen or so of other peers who on there turn are connected to a dozen or so of other peers, and so forth, creating this larger meshed network of nodes. Furthermore, although the internet may seem to serve us information instantaneously from all over the world, information still takes time to transfer across large distances.
The result is that there is no way for nodes to ensure they can come to a complete agreement on what transactions should be added to the database simply by sharing individual transactions they have seen with each other. It becomes impossible to know in what order transactions where actually submitted by users and to know what all other nodes believe to be true.
In order to help all nodes reach an agreement, we could introduce a central party that aggregates a list of transactions from all the nodes, orders them, and makes the ultimate decision on what transactions to add. Of course, that brings us back to square one, reintroducing the problems of trust in a central party which we are working so hard on to solve. That's a no-go.
This issue on how to reach consensus across a large distributed network without a central authority, is also referred to as the Byzantine Generals Problem. Solving this problem is at the heart of Bitcoin's technological innovation. The solution Bitcoin presents is, unsurprisingly, quite technical, so do not worry if this goes over your head initially. We will first discuss it at a high-level, and then go through some videos that explain it in a bit more detail.
When a user of Bitcoin wants to send bitcoin to somebody else, they will broadcast their transaction to the network (we will discuss how they do so in a later segment). Instead of nodes just adding whatever transactions they hear which would lead to disagreement across the network, they need a rule that they can follow that ensures that everybody ends up adding the same transactions.
It is also important that it is easy for us to verify that a node or group of nodes has followed the rule, when we interact with the network. We do not want to make it difficult to verify the validity of the data a node presents because that would make the auditability of bitcoins a problem. We want to be able to request information from a node on who owns what and what transactions have been sent, and be able to verify that the data is indeed representative of what the network has agreed on. The software that all the nodes are running has this rule that ensures the entire network can reach consensus and remain synchronized.
As indicated before, nodes are responsible for storing the entire history of bitcoin transactions and adding new transactions to the ledger. But they are not by default the ones that suggest what new transactions to add.
When a user of Bitcoin wants to send bitcoins to another user, he will broadcast his transaction to some of the nodes of the network. This transaction, which at this point is nothing but a potential transaction that is not yet officially confirmed on the network, gets passed along the entire network of nodes. Some of these nodes are not just nodes; they are called mining nodes or miners.
Miners listen for transactions that bitcoin users have broadcasted to the network, order them, and group these transactions together in a block. Then, they send this block of transactions to the other nodes of the network to add to their database. Below is an rough, simplified visualization of what such a block may look like, although in reality this uses code to be structured.
At this stage, we still have the same problem: how do nodes determine what block of transactions to add to their database, if all these miners just start sharing blocks? The rule states that nodes will accept a block of transactions as valid to add to their database, only if the miner can show they have spend a certain amount of work. You can sort of think of this as miners needing to show proof that they have solved a mathematical puzzle that is related to their block.
This is a little vague right now but the videos we will present shortly will go into a little bit more detail here. The core idea is that, by requesting miners for proof that they have spent a certain amount of work, it allows for the selection of which block is going to be added. With no requirement for work, nodes would be overloaded with millions of potential blocks with no way to determine which block they and all the other nodes should add, similar to when everybody would just be sharing individual transactions with each other.
Instead, because this mathematical puzzle is so difficult and requires a lot of time, the chance of two miners finding proof at the same time is very, very small. This means that the moment a block with proof is shared, it has time to propagate across the entire network as the valid block, all nodes can themselves easily verify the proof, and the network can feel confident they are all going to add the same block of transactions.
Once a node verifies the proof-of-work for the block is valid, they add it to their ledger of previously confirmed blocks. As all nodes add the block to their own ledger, the transactions in the block are confirmed on the Bitcoin network. Because this "ledger" that the nodes maintain is really just a long chain of blocks that are filled with transactions, we often refer to it as the Bitcoin blockchain. This means that transactions are stored in blocks that are chronologically linked to each other, representing the entire history of all transactions that have ever taken place.
Now, this is a somewhat simplified version of what is happening, believe it or not, and you might have a lot of unanswered questions. You might wonder, among many other things, what if two miners find a block at the same time as such that nodes receive two potentially valid blocks? Or, how do miners show proof that they have spend "work" in a way that is easy to check? What kind of "work" are we talking about?
Although the upcoming videos are also still somewhat technical, I hope they can help ground these concepts of mining blocks and consensus by proof-of-work a little more.
Takeaway There are special nodes called miners that are, like regular nodes, distributed across the world. Their sole responsibility is to collect transactions that users wish to send, order them, group them together in a block, and generate a proof of work. If they are the first to find the proof of work, they send the block of transactions to the nodes of the network, and each node adds it to their blockchain database. This proof-of-work rule is what helps a fully distributed network of nodes reach consensus on what transactions to add and secures the history of the bitcoin blockchain.
