In Part I, I covered the economics of Bitcoin. Let’s now delve into how Bitcoin works to establish trust without a central authority.
The basics of hash functions
At its core, Bitcoin relies on a very important cryptographic technique called hashing, which is performed by a hash function. This is basically a mathematical function (or formula), with an established algorithm, that operates by consensus in the cryptographic community. It takes some input data of any length and converts it to some output data with a fixed length. It is also known as a one-way function, because its entire premise in terms of providing data integrity, lies in that the output value (known as the hash, checksum or digest) can never feasibly be used to recreate the input value (the message). Hash functions must also have some useful mathematical properties to help them achieve this aim: they must be deterministic, meaning the same input must always produce the same hash (output), no matter who applies the hashing algorithm. They must be able to compute hashes very quickly. They must operate with an element of randomness, generating completely different hashes when the input data is tweaked even slightly. And of course, while it remains mathematically possible for two different messages to produce the same hash, something known as a collision, this must be statistically infeasible. Hence we can now begin to see how trust can be enforced using hashing, by virtue of being able to calculate irreversible data that cannot feasibly be tampered with.
Applications of hash functions
A well known example of a hash function is the SHA256 hashing standard. You can download this hash function along with other industry standards here (for windows) and play around with them. Simply select any file as the input and run the function to generate a fixed 65 string hexadecimal hash (in the case of SHA256). Hexadecimal here means a kind of computing number system that uses digits 0-9 and letters A-F to represent any message, hence it is of base 16 (10 digits + 6 letters). To convert hexadecimal to binary, we multiply the hexadecimal string (65) by 4 and subtract 4, to get the number of binary bits. This amounts to (65*4 – 4 = 256 bits) hence the reason why ‘256’ is in SHA256. Binary is the simplest and most common number system a computer uses, representing any message with either 1’s or 0’s (hence binary, of base 2). Most people are familiar with the decimal number system in every day life, which is based on 10, or base 10.
One common use of hash functions in desktop computing is running it as a checksum for downloads, in order to ensure the downloaded file has not been tampered with by any malicious actor. If a website provides a download with a checksum, this is an example of using hashes for checking data integrity. Simply take the download as your input and then run the relevant hashing algorithm. It should produce exactly the same hash, or checksum, as claimed. If not, then this may alert you to the file being tampered with by malicious actors. A very common application of hash functions is storing passwords. When you enter a password as a login to any website/platform, the way this is deemed as being correct is not by storing the actual passwords themselves – this would be too much of a risk. Instead, a database stores hashes of passwords against a list of user names, so that when a particular user logs on, the password hash is checked with a hashing function. Hash functions in other words, are very important to information security, and lie at the heart of crypto-currencies.
There are thousands of crypto-currencies, and many of them use different hashing algorithms for different purposes. Bitcoin was the first to introduce the proof-of-work (PoW) concept, which utilises the SHA256 hashing function to create a public ledger, blockchain, as well as for “mining” (creating) the supply of Bitcoins. As we have already seen, hash functions can be used to generate one-way hashes of input data, so one can immediately see how this can be used to sign transactions in an irreversible way, establishing a mechanism of mathematical trust.
Enter the Blockchain
Hashing may be fundamental, but on its own its not enough for proving trust. We also need a mechanism to ensure that the double spending problem is resolved, that is, what is to stop me from hashing two different transactions and double spending my money, effectively cheating the system? This is where the genius of blockchain comes into play – a public ledger of all hashed transactions backed by a network of mining nodes which perform computational work and operate under game theory to prove which transaction happened first before the other, thus guaranteeing no cheating. Each block has what is called a height, a sequential number that describes how many blocks away it is from the genesis block (block 0), which started the blockchain in 2009. The blockchain network works in such a way as to preserve the sequence of blocks being generated in a logical and seamless manner, thus ensuring a smooth public accounting of all Bitcoin transactions and eliminating the need for an intermediary to be performing this work.
The blockchain is a public chain of transaction blocks. What this means is that each ‘block’ is irreversibly linked with the previous block by hashing, so that continuity and sequencing are preserved in transaction data, and nobody can feasibly alter transactions that have already been verified by the network. This is the concept that broadly eliminates the need for a third party intermediary such as a bank. A block is a packet of data containing mostly numbers which represent a whole bunch of user transactions (which addresses paid what other addresses how many Bitcoins) along with extra details like how many transactions are ‘packaged’ into each block, the block header (a fundamental component that I’ll get to soon) and the blocksize. When Bitcoin was originally designed by Satoshi Nakamoto, he introduced a design limitation whereby the blocksize would be limited to only 1MB, thus restricting the number of transactions that could be squeezed into each block.
Bitcoin consequently underwent a ‘soft fork’ in August 2017, which meant that a backward compatible ‘software update’ to its code was agreed to by consensus in the community, and a technical workaround was introduced to new blocks by segregating some data payloads outside of the block, so that each block could handle more transactions than the original 1MB blocksize. In the same month however, part of the Bitcoin community did not agree with this decision, and underwent a ‘hard fork’, effectively splitting off from the main Bitcoin blockchain and forming a new currency, that would share the entire blockchain transaction history with Bitcoin only up until the point of fork. This new currency was called Bitcoin Cash, and instead of segregating data payloads outside the block, the team instead opted to increase the blocksize to 8MB outright in order to fit more transactions into each block. I’m not going to go too much into why this happened and who was involved, but in short, this is just the nature of decentralized technology, – not everybody agrees on a common solution, and there is always a degree of politics involved within the coding community.
Now, the most interesting aspect of each block is called the block header, and it is this piece of information that links blocks together by hashing. Inside the block header lies yet more information:
Block Version – Which version of blocks are running (this can change subject to software upgrades by the Bitcoin core development team which can tweak the block structure).
Hash of the Previous Block Header – The hash of the previous block header, using SHA256. This in turn is eventually hashed again within the new block, creating a block header hash that is passed onto the next block and so on.
Merkle Root (Root Hash) – The hash of a merkle tree using SHA256. A merkle tree is a hash tree containing all transactions in a block, hashed in sequential ways as to prevent tampering and double-spending of Bitcoins. The merkle root takes the hash of the whole data tree (remember we can hash any sized input to produce a fixed size hash output) and in this way acts as a compact cryptographic ‘summary’ of the transaction data, without needing to append the actual transaction data to the header, which would otherwise be too big.
Timestamp – A small data payload of the block timestamp, to distinguish it from other blocks being generated.
Bits – Also known as a block target. This component is aimed at miners and relates to the blockchain ‘difficulty’. I’ll explain this more when I talk about how mining works. It consists of a floating point number (without getting too technical) that is represented in a compact form to save data overhead. It is unpacked by miners into a much larger (256 bit) number for hashing.
Nonce – This is a 32-bit randomly generated number that is also used in mining, which I’ll explain next. There are (2^32 -1) possible variations of the nonce, which is a huge number.
Mining and Proof-of-Work
All of this data is contained in the block header. It is then run through a SHA256 hash function to produce a single hash which becomes the hash of the previous block header in the next block, and so on. This is how transactions ‘flow through’ the public ledger in an irreversible way. Miners do this with dedicated hardware in a process known as hashing, due to the prevalence of using hash functions. But what is mining, and why is it important to Bitcoin? It turns out that mining has much to do with, well, you guessed it: hashing.
As Bitcoin transactions are made between users (Bitcoin demand), this is algorithmically linked to a process called mining (Bitcoin supply). Mining is how the Bitcoin supply increases, steadily until an upper finite ceiling of 21 million coins is reached. After that, no more coins will be mined, meaning that Bitcoin will be endowed with the property of being scarce and in limited supply, thus always guaranteeing value against fiat currencies. It is estimated that the last coin will be mined in the year 2140, however by 2036 over 99% of all Bitcoins will have been already mined. Mining is a crucial process where transactions get verified by the network (by mining nodes – i.e. dedicated Bitcoin processors) and transactions simultaneously increase the Bitcoin supply. This works by the innovative proof-of-work (PoW) concept elaborated by Satoshi, where computational work (costing electricity) for the miner needs to be expended in order to verify transactions. Miners thus compete against each other in a cryptographic ‘game’ to verify blocks, and the first miner to verify a particular block, ‘wins’ the game and gets rewarded with Bitcoins. This is how the Bitcoin supply increases and the blockchain grows – and miners are rewarded a portion of Bitcoins for this work. But what exactly is this game?
It’s actually nothing very fancy, but rather, a very iterative and computationally intensive process. To understand it, we need to go back to the block header. When a miner takes a bunch of pending transactions and bundles them into a block, in order to verify them, he must perform the following proof-of-work (PoW) algorithm:
Take the entire block header of the block, append the nonce to the Merkle Root (which is a hash of all the transactions), re-hash the block header and compare the re-hashed value to the Bits (the target difficulty). If the re-hashed value is less than the target, then the miner gets a block reward (an amount of Bitcoins) and all the transactions in the block are verified on the network. If the re-hashed value is is more than the target, then the nonce is iterated to the next random guess, re-appended to the Merkle Root, the block header re-hashed, and the hash compared to the target again. This process of hashing continues billions and trillions of times (there are 2^32 -1 possible iterations of the nonce to be tried) until the target hash is reached. This is the computationally intensive proof-of-work algorithm that mines Bitcoins, verifies transactions and grows the blockchain.
Lets go even further, and ask why there is a target to begin with? The target is established by the Bitcoin network to automatically change every 2 weeks (or every 2016 blocks at a rate of 1 block per 10 minutes) in accordance to how the global hashing power of the Bitcoin network changes, in order to keep the generation of a Bitcoin block steady at being mined on average every 10 minutes. If the global hashing power increases (too many people are mining Bitcoins with powerful processors), then the target is lowered to maintain the rate of 1 block being mined on average every 10 minutes. This means block difficulty increases because it is harder to hash a target below this value. If too few people are mining Bitcoins, then the target is raised to maintain the rate of 1 block being mined on average every 10 minutes. This means block difficulty decreases because it is easier to hash a target above this value. This in turn encourages more mining. The variable target thus serves to maintain the rate of block generation fairly constant in accordance to how hashing power changes with time.
How do miners pick what transactions go in each block? When Bitcoin transactions are made, they first pile up as pending transactions in the memory pool, or mempool. This is a backlog of all the pending transactions on the network. Miners then pick transactions from the mempool and bundle them into blocks, where they begin the proof-of-work algorithm outlined above. Miners choose transactions based on transaction fees. While the paying of fees is optional on the Bitcoin network, those who pay higher fees have a higher probability of having their transactions processed sooner because miners are seeking to maximise profits. Those who choose not to pay any fees at all can actually run into a real risk of not having their transactions processed at all. Fees are an economic mechanism of allocating transactions. During extreme volume bottlenecks, the mempool piles up, as too many transactions are made for the hashing power of the network to handle. As a result, transactions take longer to confirm and fees rise due to competitive pressure. Many transactions, especially with low fees, are delayed sometimes up to days, while the high fee transactions are prioritised. During normal volume, the standard protocol in Bitcoin is for a transaction to be confirmed 6 times before it is officially verified. This means that 6 blocks must be mined since the first block containing the transaction is mined, which statistically reduces any chance of a double spend attack. In reality, most merchants and exchanges accept a Bitcoin transaction much sooner than 6 confirmations.
Now, what if 2 blocks are broadcast to the network at exactly the same time? How does the network ‘know’ which block is the right one? The standard taken here is that the chain with the bigger work or number of hashes is taken first, given the same difficulty level. Lower difficulty means less work. But here we are assuming the same difficulty level. So what happens is that some nodes take the first block and some take the second block and append it to their chain. Pretty soon, the process repeats and the likelihood of broadcasting the same blocks again decreases, thus consensus is found pretty quickly when a new block is mined again and appended to one of the previous chains, rendering it longer. Nodes then take the longest chain and reject the shorter ones.
The Bitcoin supply is controlled algorithmically to increase at a decreasing rate, i.e. in logarithmic fashion. I already mentioned how the Bitcoin supply increases when mining occurs. I also covered the importance of the 2 week difficulty adjustment period (or every 2016 blocks) to maintain the block generation steady on average at once per 10 minutes. Exactly how this happens is that the number of Bitcoins generated or released upon the mining of a new block, or the block reward (the Bitcoin supply), halves every 210,000 blocks. From 2009 -2011 the block reward was 50 Bitcoins per block to a miner that mined one block. From 2012-2015 the block reward halved to 25 Bitcoins per block, and from 2016 the block reward halved again to 12.5 Bitcoins per block. This will continue until all 21 million Bitcoins are mined, guaranteeing its finite supply. During the mining era, miners will make profits by both the block reward and transaction fees. During the post-mining era when all Bitcoins will have been mined, miners will still be needed by the network to ensure the irreversibility of blockchain transactions (hashing), however their income will solely come from transaction fees and no longer the block reward.
With crypto-currencies like Bitcoin, there is no concept of ‘storing’ Bitcoins on any hard drive. Bitcoin is based on public key cryptography, which means as long as you retain your private key (or seed), you can regenerate your Bitcoin ‘balance’ from any computer which hosts a Bitcoin wallet. Thus even if your computer crashes and you had a Bitcoin wallet on it, if you had your private key written down on paper, you could regenerate all your funds again on another computer and still not ‘lose’ any of them. Why is that? This is the beauty of the blockchain, which acts like an analogue of a ‘cloud’ service. Your Bitcoin ‘balance’ is actually a public record of all inputs (transactions going in) and outputs (transactions going out); every Bitcoin balance is. Some people claim that Bitcoin has a value of zero, is inherently worthless and has no intrinsic value. The truth however, is that the Bitcoin network and computational infrastructure, as outlined in this article, itself adds an intrinsic value to Bitcoin.
And that is essentially the genius of Bitcoin, – the fusion of computer science with economics.