It’s a late, Saturday night at SupremeCryptoDevHackEthDisruptCon. The clock just struck 1AM. A frenzied mob of stenchy developers are circling the snack table as the fresh batch of kombucha is being laid out, almost trampling the poor kombucha merchant in the process.
But, in the corner, there is one dev, too engrossed in her project to be distracted by the allure of that sweet, tangy, carbonated elixir.
Meet Alice - 👩💻. 👩💻is buidling the backend for a p2p savings collective to provide an open and fair financial product for millions of the financially-excluded. Well aware of the disastrous impact that crypto volatility can have on a savings group’s portfolio, the only reasonable solution is Dai. However, it’s late. The deadline for final commit is less than 8 hours away. 👩💻does not have the time nor the caffeine-reserves available to read and understand the Dai contracts. 👩💻needs an effective and secure solution asap.
Thankfully, those awesome devs at Maker have deployed their own, authenticated, EthPM package registry - overflowing with useful packages, securely stored and distributed through content-addressable URIs.
Good artists copy, great artists steal.
– somebody famous
👩💻, the resourceful artiste that she is, quickly fetches the Dai token package, and with a couple of lines of code, integrates full Dai compatibility directly into her backend.
:relief: :relief: :relief:
Now, 👩💻 has 5 minutes to catch her breath and appease that crypto bro who just won’t leave her alone until he explains his brilliant theory about how the universe is just one big merkle tree, before setting to work tidying up the front-end (as her front-end dev is across the road about 8 beers deep).
How did 👩💻 use EthPM to complete her award-winning project in so little time? Let’s find out…
What are we building?
👩💻 is building a Django backend (aka Python), using web3.py to interact with the blockchain. We won’t be using Django in this blog, but in order to reproduce the following code examples, you should be familiar with Python and have the following programs installed on your machine.
If you prefer semi-colons (aka JS), checkout
ethpm-js. But, note that the API might differ slightly in
ethpm-js from this blog post.
Before we continue…
EthPM is a tool first and foremost for developers. Your grandma is not going to be excited about installing entire smart contract systems with a single line of code. This blog post is intended to teach developers how they can start buidling with EthPM. If you have questions about what EthPM is, or why it is neato, I strongly encourage you to check out this lovely blog post by Nick Gnidan or read the EthPM Spec.
The First Rule of EthPM
- Do not install packages from untrusted registries.
The Second Rule of EthPM
- Do not install packages from untrusted registries.
Why not? Well, it is simple for anybody (yes, even the baddies) to deploy their own package registry, and publish their own malicious packages. Therefore, that
wallet package you’re about to download from the registry you saw in an
r/ethdev comment, could very well have a backdoor ready to siphon away all your ether.
Please make sure that the packages you install are sourced from a trusted registry.
That being said…
The package registry we’ll be using in this blog is mine and NOT Maker’s. While I’ve crafted the packages with tender loving care and affection, they have NOT been thoroughly audited, and should NOT be used in production. Here is proof that I control the registry used in this post.
- My ENS Name:
- My owning address:
- Keybase id: keybase.io/nickghita
- Signed proof
You can check out the package registry on our new registry explorer here.
Time to Code
Before we get to the fun stuff, some setup must happen before we can send Dai.
In this tutorial we’re going to be using the mainnet, because it’s way cooler. A few things you’ll need to complete the code examples in this post …
- A mainnet account (henceforth referenced to as
OWNER_ADDRESS) and its private key, funded with some ether and Dai. (Metamask is a good place to start if you don’t have an account)
- Another mainnet account (henceforth referenced to as
FRIEND_ADDRESS), to which you will send your Dai. (If you want your Dai returned, make sure you have the private key for this address, or it belongs to a friend you trust).
Run these commands in your terminal to setup your virtual environment and start your IPython session…
# Create a directory >>> mkdir ethpm-blog && cd ethpm-blog # Check Python version, make sure it's 3.6 or above >>> python --version Python 3.6.0 # Create your virtual environment >>> python -m venv venv # Activate your virtual environment >>> source venv/bin/activate # Install web3.py >>> pip install web3==5.0.0a4 && pip install ipython # Start your IPython session >>> ipython
Now let’s setup some of the account variables we’ll be using…
(hint: You can find the private key for a Metamask account by clicking on
"Account Details" ->
"Export Private Key")
>>> from eth_account import Account >>> from web3.auto.infura import w3 >>> from web3.middleware import construct_sign_and_send_raw_middleware # Setup Accounts # Insert your private key here >>> OWNER_PRIV_KEY = "0x123456..." >>> OWNER_ADDRESS = Account.privateKeyToAccount(OWNER_PRIV_KEY).address # Insert the address you're sending Dai to here >>> FRIEND_ADDRESS = "0x321...." # Settings to automatically build & sign transactions >>> signing_middleware = construct_sign_and_send_raw_middleware(OWNER_PRIV_KEY) >>> w3.middleware_onion.add(signing_middleware) >>> w3.eth.defaultAccount = OWNER_ADDRESS # Double check connection and chainID >>> w3.isConnected() True # '1' == mainnet >>> w3.net.version '1' # We must manually enable the package management API, as it's API is not yet stable >>> w3.enable_unstable_package_management_api() # Confirm you're all set up and ready to roll >>> w3.pm <web3.pm.PM at 0x....>
Fetch the Dai package
To use a package, we must first obtain its manifest (ex. the JSON file containing all the juicy contract assets), located at a content-addressed URI (a tamper-proof URI that contains a hash of the stored data). The most secure way to fetch a package’s manifest URI is by fetching it from an authenticated, on-chain registry (remember, whose owner you trust!).
# Set the active registry on the PM instance >>> w3.pm.set_registry("snakecharmers.eth") # Fetch the Dai package >>> dai_pkg = w3.pm.get_package("dai", "1.0.0") # Confirm you have the Dai package >>> dai_pkg <Package dai==1.0.0>
Send Dai to another account
OK, so we have a package. What do now?
Achtung! In order for this to work,
OWNER_ADDRESS must already own Dai! The easiest way to acquire Dai is from the Oasis DEX.
(For those of you early to this EthPM party, I have some spare Dai to give away. Ping me (
"Dai Plz":<your github account>:<your favorite song>:<your ethereum mainnet address> on our gitter channel and I’ll send you a some Dai to experiment with. Offer only valid while supplies last).
# Fetch the contract instance of the mainnet Dai deployment >>> dai = dai_pkg.deployments.get_instance("DSToken") # Let's check the initial balances >>> dai.functions.balanceOf(OWNER_ADDRESS).call() 100000000000000000000 # == 100 Dai :) >>> dai.functions.balanceOf(FRIEND_ADDRESS).call() 0 # == 0 Dai :( # Build & send the tx to send Dai to our friend >>> tx_hash = dai.functions.push(FRIEND_ADDRESS, web3.toWei(1, 'ether')).transact() # Wait for the tx to be mined >>> tx_receipt = w3.eth.waitForTransactionReceipt(tx_hash) # Let's see if it did the thing... >>> dai.functions.balanceOf(OWNER_ADDRESS).call() 99000000000000000000 # == 99 Dai :) >>> dai.functions.balanceOf(FRIEND_ADDRESS).call() 1000000000000000000 # == 1 Dai :) # It did the thing!!
- No copy and pasting ABI monstrosities or entire libraries of source code
- No more folders and folders of outdated assets cluttering your project code.
- No compiling contracts in Remix, copying the bytecode over to your project. . .
“Oh wait, I forgot to check optimize, F***, now I have to start all over again.”
– Another copy and paste session later…
“Oh wait, I’m supposed to use
A tidy, versioned, package makes it effortless to interact with, manage and share your contracts as they develop and evolve, on-chain or locally.
EthPM turns your contracts into packages, eliminating error-prone and tedious copy and pasting and making it fast & simple to …
- Deploy contract instances
- Interact with deployed contracts
But wait! There’s more!
What else can EthPM do?
But mainnet is expensive . . .
There’s no such thing as free Eth (except on the testnets). So, what if 👩💻 wants to send Dai on Kovan for her earth-shattering demo without breaking the bank? No problem! All she needs is a
web3 instance connected to the Kovan testnet.
>>> from web3.auto.infura.kovan import w3 as kovan_w3 # Returns a new Package instance set with the Kovan web3 instance >>> kovan_dai_pkg = dai_pkg.set_w3(kovan_w3) # Fetch the Dai deployment on Kovan, and proceed as before >>> kovan_dai = kovan_dai_pkg.deployments.get_instance("DSToken") # Observe that kovan_dai is a different instance than mainnet >>> kovan_dai.address '0xC4375B7De8af5a38a93548eb8453a498222C4fF2' >>> dai.address '0x89d24A6b4CcB1B6fAA2625fE562bDD9a23260359' # Now, you can do all the things with `kovan_dai` that we did with `dai` # assuming that you've repeated the setup on your `kovan_w3` instance
Notice how you don’t need a new package to access the deployment on a different chain. A package is chain-agnostic and can contain contract deployments on any chain: mainnet, testnet, or private.
Deploy a fresh DSToken
What if 👩💻 needs the ability for each user to deploy their own instance of
# Fetch the Dai contract factory >>> ds_token_factory = dai_pkg.get_contract_factory("DSToken") # Deploy a new Dai instance for "user_x" >>> tx_hash = ds_token_factory.constructor(b'user_x_dai').transact() # Wait for the tx to be mined >>> tx_receipt = w3.eth.waitForTransactionReceipt(tx_hash) # Voila! The address of "user_x"'s brand new DSToken >>> tx_receipt.contractAddres "0x123abc......"
Deploy a DEX
Under the heavy influence of maté tea and the near-toxic body-odor emerging from the table next to her, 👩💻 experiences a brilliant revelation of economic understanding unparalleled since the enlightenment, understanding that if anybody can dynamically deploy their own DEX, on-demand, the ramifications could usher in a new economic era of unbridled prosperity. Easier said than done? Nope, there’s a package for that!
# Fetch Maker's Simple-Market package >>> simple_market_pkg = w3.pm.get_package("simple-market", "1.0.0") # Fetch Simple-Market contract factory >>> simple_market_factory = simple_market_pkg.get_contract_factory("SimpleMarket") # Build, sign & send simple-market deployment tx >>> tx_hash = simple_market_factory.constructor().transact() # Wait for tx to be mined >>> tx_receipt = w3.eth.waitForTransactionReceipt(tx_hash) # Voila! The address of your newly deployed DEX >>> tx_receipt.contractAddress "0xjfk456..."
No matter how complex a smart contract system is, it can be packaged up and distributed with EthPM, making it trivial for developers all around the world to start building on top of your contracts (hint hint, anybody developing a protocol who wants to make it simple for other devs to build on top).
Deploy an unhackable wallet contract
What if 👩💻hears about a revolutionary package registry with a
wallet package that is literally impossible to hack! Well — 👩💻is no dummy, and obviously wouldn’t use that package without verifying the package contents first or doing due diligence on the registry owner.
Remember the first rule of EthPM.
Do not install packages from untrusted registries.
Deploy your own package registry
If you are an auditor, consultant, protocol developer, or simply a cool and interesting person, you might be wondering how you can obtain a package registry to share your contract packages with the world? Well, I’m glad you asked!
# Please be sure to save this address! # This will automatically set your freshly deployed registry as the active registry on # your w3.pm instance, but please save your contract address for future reference. >>> w3.pm.deploy_and_set_registry() "0x876rst..." # Your new registry address!
deploy_and_set_registry method above automatically deploys an instance of the
Vyper Reference Registry implementation. PLEASE NOTE: THEY VYPER IMPLEMENTATION IS CURRENTLY PENDING AN AUDIT, AND SHOULD NOT BE USED IN PRODUCTION JUST YET There’s also a
Solidity Reference Registry implementation you should check out if interested - and an easy to deploy integration with
web3.py coming soon.
Publish a package to your registry
Crafting a package for your own contracts is beyond the scope of this blog. If you’re an ambitious kinda person, I suggest checking out the builder tool as a good place to start. But, if you simply want to try publishing any ol’ package to your registry, let’s do that with the
owned package, version
1.0.0, stored at
ipfs://QmbeVyFLSuEUxiXKwSsEjef6icpdTdA4kGG9BcrJXKNKUW. You can check it out here.
>>> w3.pm.release_package( "owned", # package name "1.0.0", # package version "ipfs://QmbeVyFLSuEUxiXKwSsEjef6icpdTdA4kGG9BcrJXKNKUW", # content-addressed URi )
Now you can check out your newly published package on the Registry Explorer, or with the following command.
>>> w3.pm.get_all_package_names() ('owned',) >>> w3.pm.get_all_package_releases('owned') (('1.0.0', 'ipfs://QmbeVyFLSuEUxiXKwSsEjef6icpdTdA4kGG9BcrJXKNKUW'),)
The Registry Explorer
We’ve created the EthPM Registry Explorer for your enjoyment. With this tool, you can browse registries across the mainnet, Ropsten, Rinkeby, and Kovan. It supports both the Vyper Reference Registries and the Solidity Reference Registry. So, if you’ve just deployed your registry with
w3.pm.deploy_and_set_registry() as shown above, check it out on the explorer. The explorer also supports ENS on the mainnet.
A final word on trust
EthPM is a tool for developers, designed to make your development workflow faster and easier. With the recent flurry of malicious packages showing up in traditional package managers, it’s entirely likely that there will be malicious packages out there in the EthPM ecosystem. The good news is that with EthPM, you don’t rely on an open, centralized package manager. As long as you control your keys, you control your registry. Please, do your due diligence and only use registries who are maintained by owners you trust.
Have questions? Want answers? Come join us in our Gitter channel
Feel free to ping me directly -
- EthPM Specification
- EIP 1123: Ethereum Smart Contract Packaging Standard
- ERC 1319: Smart Contract Package Registry Interface
- Blog: EthPM - Reusable Smart Contract Packages
- py-ethpm: core python library
- ethpm-go: core go library
- escape truffle: solidity package registry implementation
- web3.py/pm: web3.py package management api