Getting Started with Ethereum Dev

I’ve written a few blog posts about setting up local networks for BitCount, BitMessage and Hyperledger Fabric, and I started out with the intent to write a similar post for Ethereum.  I’ve found that for Ethereum, with the use of Make and Docker, it’s a pretty straightforward process.

The first step is to check out the Ethereum codebase and build the tools:

mkdir -p $GOPATH/src/
cd $GOPATH/src/
git clone
cd go-ethereum
go install -v ./...
cd $GOPATH/bin


./geth --pprof

That’s it!

To run unit tests, just (for example) do the following:

go test -v -cpu 4 ./eth

This is all explained in the git repo’s readme file here.

Setting up a local network is equally straightforward.  You can read up on it here.  I won’t go through all the details but if you’re interested in some makefiles and Docker build images just send me a note.

One interesting aspect of Ethereum is that they expose their P2P protocol, so you can use Ethereum to build your own peer-to-peer networks.  This can be useful for example for building distributed services that need to run in conjunction with the Ethereum network (but don’t necessarily need to make use of the Ethereum blockchain or EVM).  You can read about Etherium’s P2P capabilities here.

(This will be the subject of a future blog.)


Blockchain 102

BitCoin is currenty running at the threshold of its ability to keep up with users’ transactions, and there are two competing proposals for addressing how this can be fixed.  (If you don’t know the background, read my previous post.)

One option has been proposed by the BitCoin core development team.  It involves striping some information out of each BitCoin transaction and storing it “off chain” – this will result in a smaller transaction footprint in each block, and therefore more capacity for transactions within the current block size.  The proposal also allows for larger blocks, and the potential to add “secondary chains”.  This proposal is called “Segregated Witness”, or SegWit.

The second option is supported mainly by the BitCoin miners, and involves increasing the size of the blocks (to allow more transactions) and also the capability for miners to increase the block size and transaction fees in the future.  Ths option is called “BitCoin Unlimited”, or BU.  This option is not supported by many stakeholders because they see this as granting too much control to the miners, and reducing the “democratic” and decentralized nature of BitCoin.

One wrinkle is that a number of the large miners have developed a (panted) optimization in the block hash calculation (the so called “proof of work”) that gives them about a 20% advantage in computing new blocks for the Blockchain.  BU will entrench this competitive advantage, but SegWit includes some provisions to neutralize it.

(As an aside, other cryto-currencies and blockchain-based technologies, such as Ethereum (more about this in a future blog post), are selecting alternate algorithms for “proof of work” to try to avoid some of the centralization that has occurred in the BitCoin network.  But more about this in a future post.)

This drama is exploding all over the Internet, sub-Reddits and discussion boards everywhere.  But it is illustrative of the nature of the BitCoin network.

Both options have been implemented and made available to BitCoin miners and nodes, and the stakeholders can implement either option and “vote” as to their preference for the future of the BitCoin network.  Some of the lager miners have threatened to unilaterally implement BU and force the rest of the network to get in line.  However the BitCoin network works based on the concept of “consensus”, and if there is no consensus, then the network won’t operate.  If the nodes don’t accept blocks created by the miners, the new blocks won’t get distributed and they won’t be part of the common Blockchain ledger.  At worst the network will “split” and there will be 2 separate BitCoins.

What does the future hold for BitCoin?


Blockchain 101

There’s a drama unfolding in the Bitcoin community right now!  It’s interesting and instructive, and I’ll blog about it in my next post.  Today I’ll go over some Blockchain 101 (actually BitCoin 101) to set some background for the drama of the next post.

The terms BitCoin and Blockchain are often used synonymously these days, but:

A Blockchain is a secure, unalterable, shared ledger.  A Blockchain consists of a series of blocks that are each “signed” with a secure stamp (for the technically minded this is a Hash, a which can mathematically demonstrate that the contents of the block have not been altered).  The signature of each block includes the signature of the previous block, hence the blocks are linked together in a chain, and one block can’t be altered without having to update all subsequent blocks.

BitCoin is a digital currency, that uses a Blockchain as the underlying structure to record transactions.  Each block in BitCoin’s Blockchain contains a set of transactions that represent a transfer of Bitcoin from one party to another.  BitCoin transactions are secured wth strong digital signatures, and the blocks within BitCoin’s blockchain are secured by placing constraints around the block signature (or Hash) that makes it extremely difficult to calculate.  In fact this difficulty is adjusted based on the size of the BitCoin network, so the larger the network (i.e. the more computing power the network can bring to bear) the more difficult it is to construct a block on the BitCoin Blockchain.  The ability to compute the Hash successfully requires a large amount of computing resources, and this is known as the Proof of Work.

The BitCoin network consists of a series of Nodes and Miners.  (Miners compute the new blocks in the Blockchain, and Nodes transmit the new blocks to all parties on the network.)  No one “owns” or “controls” BitCoin – everyone who participates in the network keeps their own copy of the Blockchain.  The Miners and Nodes (and other parties, such as BitCoin Exchanges and BitCoin users, who use Wallet applications to connect to the network) agree on the set of rules that constitutes a valid Blockchain, and will only share transactions and blocks that meet this criteria.  (The difficulty level of the block’s Hash is one such rule.)

This is called Consensus, and it is one of the most powerful aspects of BitCoin.  Consensus means that no one party can take over and control the BitCoin network, because the rest of the parties on the network won’t cooperate, and the cooperation of all parties is required for the BitCoin network to operate.  (Remember this for the BitCoin drama coming up in the next post.)

To summarize:

  • Blockchains consist of a “chain” of blocks that are “signed” by cryptographic hashes
  • Each block contains BitCoin transactions, that are protected by strong ryptography
  • BitCoin defines rules that define the “consensus” of what constitutes a valid blockchain, including a strong “proof of work” for creating each block
  • Each participant in the BitCoin network maintains their own copy of the Blockchain, and new transactions and blocks are shared amongst the particpants on a peer-to-peer network
  • The participants in the network will only share new transactions and blocks that follow these rules

With the rise in popularity of BitCoin, there are some weaknesses in the architecture that are starting to become apparent.  The first is the size of the Blockchain, which has reached 100G in size and is growing at about 4G per month.  Each participant in the network has to maintain this Blockchain, as well as support the network bandwidth to communicate the new blocks and transactions.  The second issue is the transaction throughput that the BitCoin network can sustain – due to limitations on the size of each block (one of the rules of “Consensus”), the block can hold a maximum of about 1000 transactions.  Since the network is constrained to produce a new block about every 10 minutes (another of the “consensus” rules, controlled by the “difficulty” of computing the block’s signature) this places a ceiling on the maximum transactions that BitCoin can handle.

There are a couple of alternatives on how to address these limitations, and the BitCoin community is divided on the path to take!

In my next post I’ll talk about the “drama” surrounding this, and I’ll also talk about some other technologies and applications that are being built today around Blockchains and Blockchain technology.


Getting Started with Hyperledger Part 2 – Fabric Development

I wrote a post describing how to get a sample HyperLedger Fabric demo up and running, but I generally like to work from first principles, so in this post I am going to describe how to checkout the source for the Hyperledger Fabic core components and build the docker images from scratch.

Setup Development Environment

First of all there are a couple of pre-requisites:

The source code is managed on “Gerrit“, and you can checkout the code from the Git repo, but if you want to be a “committer” and be able to contribute to Fabric development, so will need to setup a Linux Foundation ID and check out the code under this ID.  The process to do this is described here.

You will also need to install all the pre-requisites on your local (or on your developer VM), this process is described here.  (There is also a good blog post here on IBM’s Community Blog, however it’s a bit dated and the pre-req setup references old versions.  However the blog is still a good read.)

Checkout and Compile Code

Now, assuming you have a Linux Foundation ID, you can check out the code as follows:

First of all make sure you are in your “Go” source tree:

# create directory to checkout Hyperledger code
cd ~
mkdir go
cd go
export GOPATH=$PWD
echo $GOPATH

mkdir -p src/
cd src/

Now checkout the following repositories (replace “gerritid” with your own id):

# check out the fabric repo
cd $GOPATH/src/
git clone ssh:// && scp -p -P 29418 fabric/.git/hooks/

# check out the fabric CA
git clone ssh://

(If you take a look at the Gerrit site you can see a list of the other Hyperledger projects, including the SDK’s, base images, etc.)

You can now build the base Fabric docker images.  You can just run a “make dist-clean all” and it will do everything, but let’s take it step by step.

First do a “dist-clean” to make sure your workspace is clean and the pre-requisites are all there:

# clean the local repo
cd $GOPATH/src/
make dist-clean

You shouldn’t see any errors.

There are several steps to build and test the docker images – first compile the code for the peer and orderer processes:

# compile the peer and orderer processes
make native

If you get any errors you may have missed a dependency.  Just google the error message and stacktrace will tell you what to do!

Since this takes awhile (the first time around it has to download some base images) you can take a look at the source code while you wait – the top-level code is in the peer and orderer packages, and shared code is in common, core, etc.  I’ll talk about this some more in a later blog.

Next build the docker images (peer and orderer):

# build the docker images
make docker
docker images
docker ps

You should have a few Docker images, but nothing running yet.

At this point you can run the unit tests, this takes awhile so make yourself a coffee and put your feet up:

# run the unit tests
make linter
make unit-test

While the unit tests are running open another command window and run “docker ps”, you will see docker containers running as the unit tests execute.  (For me the test run for awhile and then start to fail, seems like a stability issue more than anything else.)

Next run Behave (behaviour-driven development framework in Python):

# run behave
make behave

(This will be the subject of a more detailed future blog.)

You can also build the CA image as follows:

# clean the local repo
cd $GOPATH/src/
make docker

You should now have Docker images for the peer, order and ca containers (as well as a whole collection of other images created by the build and unit test processes).

What’s Next?  Contribute to the Hyperledger Project  You can read up on how to contribute, here.  And I’ll talk about it in more detail in a future blog.