Blockchain 102

BitCoin is currenty running at the threshold of its ability to keep up with users’ transactions, and there are two competing proposals for addressing how this can be fixed.  (If you don’t know the background, read my previous post.)

One option has been proposed by the BitCoin core development team.  It involves striping some information out of each BitCoin transaction and storing it “off chain” – this will result in a smaller transaction footprint in each block, and therefore more capacity for transactions within the current block size.  The proposal also allows for larger blocks, and the potential to add “secondary chains”.  This proposal is called “Segregated Witness”, or SegWit.

The second option is supported mainly by the BitCoin miners, and involves increasing the size of the blocks (to allow more transactions) and also the capability for miners to increase the block size and transaction fees in the future.  Ths option is called “BitCoin Unlimited”, or BU.  This option is not supported by many stakeholders because they see this as granting too much control to the miners, and reducing the “democratic” and decentralized nature of BitCoin.

One wrinkle is that a number of the large miners have developed a (panted) optimization in the block hash calculation (the so called “proof of work”) that gives them about a 20% advantage in computing new blocks for the Blockchain.  BU will entrench this competitive advantage, but SegWit includes some provisions to neutralize it.

(As an aside, other cryto-currencies and blockchain-based technologies, such as Ethereum (more about this in a future blog post), are selecting alternate algorithms for “proof of work” to try to avoid some of the centralization that has occurred in the BitCoin network.  But more about this in a future post.)

This drama is exploding all over the Internet, sub-Reddits and discussion boards everywhere.  But it is illustrative of the nature of the BitCoin network.

Both options have been implemented and made available to BitCoin miners and nodes, and the stakeholders can implement either option and “vote” as to their preference for the future of the BitCoin network.  Some of the lager miners have threatened to unilaterally implement BU and force the rest of the network to get in line.  However the BitCoin network works based on the concept of “consensus”, and if there is no consensus, then the network won’t operate.  If the nodes don’t accept blocks created by the miners, the new blocks won’t get distributed and they won’t be part of the common Blockchain ledger.  At worst the network will “split” and there will be 2 separate BitCoins.

What does the future hold for BitCoin?


Blockchain 101

There’s a drama unfolding in the Bitcoin community right now!  It’s interesting and instructive, and I’ll blog about it in my next post.  Today I’ll go over some Blockchain 101 (actually BitCoin 101) to set some background for the drama of the next post.

The terms BitCoin and Blockchain are often used synonymously these days, but:

A Blockchain is a secure, unalterable, shared ledger.  A Blockchain consists of a series of blocks that are each “signed” with a secure stamp (for the technically minded this is a Hash, a which can mathematically demonstrate that the contents of the block have not been altered).  The signature of each block includes the signature of the previous block, hence the blocks are linked together in a chain, and one block can’t be altered without having to update all subsequent blocks.

BitCoin is a digital currency, that uses a Blockchain as the underlying structure to record transactions.  Each block in BitCoin’s Blockchain contains a set of transactions that represent a transfer of Bitcoin from one party to another.  BitCoin transactions are secured wth strong digital signatures, and the blocks within BitCoin’s blockchain are secured by placing constraints around the block signature (or Hash) that makes it extremely difficult to calculate.  In fact this difficulty is adjusted based on the size of the BitCoin network, so the larger the network (i.e. the more computing power the network can bring to bear) the more difficult it is to construct a block on the BitCoin Blockchain.  The ability to compute the Hash successfully requires a large amount of computing resources, and this is known as the Proof of Work.

The BitCoin network consists of a series of Nodes and Miners.  (Miners compute the new blocks in the Blockchain, and Nodes transmit the new blocks to all parties on the network.)  No one “owns” or “controls” BitCoin – everyone who participates in the network keeps their own copy of the Blockchain.  The Miners and Nodes (and other parties, such as BitCoin Exchanges and BitCoin users, who use Wallet applications to connect to the network) agree on the set of rules that constitutes a valid Blockchain, and will only share transactions and blocks that meet this criteria.  (The difficulty level of the block’s Hash is one such rule.)

This is called Consensus, and it is one of the most powerful aspects of BitCoin.  Consensus means that no one party can take over and control the BitCoin network, because the rest of the parties on the network won’t cooperate, and the cooperation of all parties is required for the BitCoin network to operate.  (Remember this for the BitCoin drama coming up in the next post.)

To summarize:

  • Blockchains consist of a “chain” of blocks that are “signed” by cryptographic hashes
  • Each block contains BitCoin transactions, that are protected by strong ryptography
  • BitCoin defines rules that define the “consensus” of what constitutes a valid blockchain, including a strong “proof of work” for creating each block
  • Each participant in the BitCoin network maintains their own copy of the Blockchain, and new transactions and blocks are shared amongst the particpants on a peer-to-peer network
  • The participants in the network will only share new transactions and blocks that follow these rules

With the rise in popularity of BitCoin, there are some weaknesses in the architecture that are starting to become apparent.  The first is the size of the Blockchain, which has reached 100G in size and is growing at about 4G per month.  Each participant in the network has to maintain this Blockchain, as well as support the network bandwidth to communicate the new blocks and transactions.  The second issue is the transaction throughput that the BitCoin network can sustain – due to limitations on the size of each block (one of the rules of “Consensus”), the block can hold a maximum of about 1000 transactions.  Since the network is constrained to produce a new block about every 10 minutes (another of the “consensus” rules, controlled by the “difficulty” of computing the block’s signature) this places a ceiling on the maximum transactions that BitCoin can handle.

There are a couple of alternatives on how to address these limitations, and the BitCoin community is divided on the path to take!

In my next post I’ll talk about the “drama” surrounding this, and I’ll also talk about some other technologies and applications that are being built today around Blockchains and Blockchain technology.


Setting up a Bitcoin Development Environment and Test Network using Docker

Blockchain is a hot technology these days, and Bitcoin is the original implementation.  As a blockchain professional it is handy to be able to setup and run a Bitcoin network from scratch – checking out and compiling the code from Github, compiling and installing the code into a Docker container, and then running instances of the container to setup a small, local test network.

There are a number of projects you may want to explore (Bitcoin, Bitmessage, BigchainDB, Etherium, Hyperledger Fabric, etc.) and Docker allows you to setup multiple development environments without filling up your host with all the dependencies.

You can checkout the Bitcoin code from Github here, and I have setup a small project to build and run the Docker container here.  (Shoutout to Gerald and this blog post, which this project is based on.)

There are instructions to build and run the Bitcoin Docker image and test network in the Readme file in GitHub, so I won’t repeat everything here.  I’ll just explain a few things that aren’t covered in the readme.

The first step is to build a base Docker image that contains all the dependencies to compile and run Bitcoin – the Dockerfile is here.  It uses Ubuntu as a base image and adds all Bitcoin dependencies.

The second step is to run this base image and use it to compile the Bitcoin source.  The Bitcoin source is located on the host, and the directory is mounted in the Docker container using:

docker run ... -v $(BITCOIN_SRC):$(BITCOIN_SRC) -w $(BITCOIN_SRC) ...

(The location of the BITCOIN_SRC is configured in the Makefile, and you can edit this if you checkout your code in a different location.)

In the above, “-v” mounts the directory, and “-w” sets the current directory for running “make”.

Once the Bitcoin code is compiled and installed into the container, a second snapshot is saved using:

docker commit ...

… to save a new image.

Two instances of this image (“bob” and “alice”) are now started to run the local Bitcoin network:

docker run -d -it -p 18444:18444 -p 18332:18332 --name=alice --hostname=alice
docker run -d -it -p 19444:18444 -p 19332:18332 --name=bob --hostname=bob

Each image runs an instance of bitcoind, and the daemon is setup to run a local network (i.e. not connected to the Bitcoin production or test network):

bitcoind -regtest -daemon -printtoconsole

Commands can be invoked in each bitcoind process by executing bitcoin-cli in each Docker image:

# the following command is setup within the Makefile
docker exec alice bitcoin-cli -regtest $(bccmd)

# ... so to send a command to "alice" you simply run (for example to generate blocks):
make alice_cmd bccmd="generate 10"

It’s pretty simple!  If you have any questions please feel free to email me at