Introduction to Cross Consensus Messaging (XCM)
How to use the slides - Full screen (new tab)
Introduction to Cross-Consensus Messaging (XCM)
Core Concepts, Terms, and Logic
Notes:
Pre-requisites
- FRAME (Storage Items, Dispatchables, Event, Errors, etc.)
- Polkadot & parachains conceptually
- Assets (NFTs and fungibles)
---v
At the end of this lecture, you will be able to:
- Define the concepts, syntax, and terms of XCM
- Navigate existing resources that relate to XCM
- Differentiate between XCM and message-passing protocols like XCMP
Cross-chain use cases
Why would we want to perform operations on different blockchains?
Notes:
EXERCISE: ask the class to raise hands and postulate on generally what one might do. We are expecting them to say transfers, but there are so many other things you could do, so many more problems worth solving with cross-chain:
- One contract calling another contract
- Credential checking
- Voting
---v
๐ฌ Some Concrete Use-cases
- Cross-consensus asset transfers
- Execute platform-specific actions such as governance voting
- Enables single use-case chains
- Collectives
- Identity chains
Notes:
While the goal of XCM is to be general, flexible and future-proof, there are of course practical needs which it must address, not least the transfer of tokens between chains. We need a way to reason about, and pay for, any required fees on the receiving CS. Platform-specific action; for example, within a Substrate chain, it can be desirable to dispatch a remote call into one of its pallets to access a niche feature. XCM enables a single chain to direct the actions of many other chains, which hides the complexity of multi-chain messaging behind an understandable and declarative API.
XCM is a language for communicating intentions between consensus systems.
---v
Consensus systems
A chain, contract or other global, encapsulated, state machine singleton.
It does not even have to be a distributed system, only that it can form some kind of consensus.
Notes:
A consensus system does not necessarily have to be a blockchain or a smart contract. It can be something that already exists in the Web 2.0 world, such as an EC2 instance in an AWS server. XCM is Cross-Consensus since it's much more than cross chain.
---v
โ๏ธ A Format, not a Protocol
XCM is a messaging format.
It is akin to the post card from the post office.
It is not a messaging protocol!
A post card doesn't send itself!
Notes:
It cannot be used to actually "send" any message between systems; its utility is only in expressing what should be done by the receiver. Like many aspects core to Substrate, this separation of concerns empowers us to be far more generic and enable much more. A post card relies on the postal service to get itself sent towards its receivers, and that is what a messaging protocol does.
The transport layer concerns itself with sending arbitrary blobs, it doesn't care about the format. A common format has its benefits though, as we'll see next.
---v
Versioning
XCM is a versioned language.
It's currently in version 3.
What goes in each version is defined via an RFC process.
---v
Terminology: XCMs
XCM, Cross-Consensus Messaging, is the format.
An XCM is a Cross-Consensus Message.
It's not called an XCM message,
the same way it's not called an ATM machine.
๐ฌ Why not native messages?
Drawbacks of relying on native messaging or transaction format:
- Native message format changes from system to system, it also could change within the same system, e.g. when upgrading it
- Common cross-consensus use-cases do not map one-to-one to a single transaction
- Different consensus systems have different assumptions e.g. fee payment
Notes:
- A system which intends to send messages to more than one destination would need to understand how to author a message for each. On that note, even a single destination may alter its native transaction/message format over time. Smart contracts might get upgrades, blockchains might introduce new features or alter existing ones and in doing so change their transaction format.
- Special tricks may be required to withdraw funds, exchange them and then deposit the result all inside a single transaction. Onward notifications of transfers, needed for a coherent reserve-asset framework, do not exist in chains unaware of others. Some use-cases don't require accounts.
- Some systems assume that fee payment had already been negotiated, while some do not. It's up to the interpreter to interpret the intention how it makes sense.
---v
Message format changes
---v
Message format changes
---v
Message format changes
Notes:
XCM abstracts away the actual on-chain operation that will be called, which lets the recipient redirect calls to always make them valid.
---v
No one-to-one mapping
Notes:
You might want to withdraw some assets and deposit some amount to one account and another to another. Using transactions, you'd have to send many messages to achieve this.
---v
Different assumptions
Notes:
Different systems have different assumptions. Using native messages, you'd have to tailor your messages to all systems you want to message.
Four 'A's
XCM assumes the following things from the underlying environment.
- Agnostic
- Absolute
- Asynchronous
- Asymmetric
Notes:
The 4 'A's are assumptions XCM makes over the transport protocol and overall the environment where these messages are sent and processed.
---v
Agnostic
XCM makes no assumptions about the nature of the Consensus System between which messages are being passed.
Notes:
XCM is not restricted to Polkadot, it's a language that can be used for communication between any systems. For example, EVM-chains or Cosmos hubs.
---v
Absolute
XCM assumes that the environment guarantees delivery, interpretation, and ordering of messages.
Notes:
The message format does not do much about the message possibly not being delivered. In IBC, for example, you factor in fallibility of the transport protocol into your messages.
---v
Asynchronous
XCMs crossing the barrier between a single consensus system cannot generally be synchronous.
XCM in no way assume that the sender will be blocking on messages.
Notes:
You can't just block execution in the middle of a block, it has to be asynchronous. Different systems have different ways of tracking time. No assumption of blocking for sender/receiver.
Generally, consensus systems are not designed to operate in sync with external systems. They intrinsically need to have a uniform state to reason about and do not, by default, have the means to verify states of other consensus systems. Thus, each consensus system cannot make any guarantees on the expected time required to deliver results; doing so haphazardly would cause the recipient to be blocked waiting for responses that are either late or would never be delivered, and one of the possible reasons for that would be an impending runtime upgrade that caused a change in how responses are delivered.
---v
Asymmetric
XCM doesn't assume there'll be messages flowing in the other direction.
If you want to send responses, you have to make it explicitly.
Notes:
There are no results or callbacks. Any results must be separately communicated to the sender with an additional message. The receiver side can and does handle errors, but the sender will not be notified, unless the error handler specifically tries to send back an XCM that makes some sort of XCM that notifies status back to the origin, but such an action should be considered as constructing a separate XCM for the sole purpose of reporting information, rather than an intrinsic functionality built into XCM. XCM is a bit like REST. XCMP is a bit like TCP/IP but not quite. Analogies can often hurt more than they help.
๐ Locations in XCM
Before sending a message to another system, we need a way to address it.
Notes:
XCM defines a Location
type that acts as a URL for consensus systems.
The Location
type identifies any single location that exists within the world of consensus.
Representing a scalable multi-shard blockchain such as Polkadot, an ERC-20 asset account on a parachain, a smart contract on some chain, etc.
It is usually represented as a location relative to the current consensus system.
Relative locations are easier to handle due to the fact that the network structure can change.
Locations don't define the actual path to get there, just a way of addressing.
---v
Interior locations
Given two consensus systems, A and B. A is interior to B if a state change in A implies a state change in B.
Notes:
An example, a smart contract in Ethereum would be interior to Ethereum itself.
---v
Location hierarchy
Notes:
Locations form a hierarchy using the interior relation.
---v
Location Representation
#![allow(unused)] fn main() { struct Location { parents: u8, junctions: Junctions, } }
#![allow(unused)] fn main() { enum Junction { Parachain(u32), AccountId32 { id: [u8; 32], network: Option<NetworkId> }, PalletInstance(u8), GeneralIndex(u128), GlobalConsensus(NetworkId), ... } }
Notes:
Right now Junctions are limited to 8 because of stack space. We also don't expect Junctions being more than 8 levels deep.
It's perfectly possible to create locations that don't point anywhere.
---v
Network Id
#![allow(unused)] fn main() { enum NetworkId { ByGenesis([u8; 32]), ByFork { block_number: u64, block_hash: [u8; 32] }, Polkadot, Kusama, Westend, Rococo, Wococo, Ethereum { chain_id: u64 }, BitcoinCore, BitcoinCash, } }
Notes:
Junctions are ways to descend the location hierarchy
---v
Text notation
#![allow(unused)] fn main() { Location { parents: 1, interior: Parachain(50) } }
-->
../Parachain(50)
Notes:
This notation comes from an analogy to a file system.
---v
Universal Location
The Universal Location is a theoretical location. It's the parent of all locations which generate their own consensus. It itself has no parents.
---v
Universal Location
Notes:
We can imagine a hypothetical location that contains all top-level consensus systems.
---v
Absolute locations
#![allow(unused)] fn main() { pub type InteriorLocation = Junctions; }
Sometimes, absolute locations are necessary, e.g. for bridges.
They don't have parents.
The first junction has to be a GlobalConsensus
.
Notes:
To write an absolute location, we need to know our location relative to the Universal Location.
---v
What are Location
s used for?
- Addressing
- Origins
- Assets
- Fees
- Bridging
---v
Cross-Chain Origins
When a receiver gets an XCM, a Location
specifies the sender.
This Location
is relative to the receiver.
Can be converted into a pallet origin in a FRAME runtime
Used for determining privileges during XCM execution.
Notes:
Reanchoring:
Since Location
s are relative, when an XCM gets sent over to another chain, the origin location needs to be rewritten from the perspective of the receiver, before the XCM is sent to it.
Location Examples
---v
Sibling parachain
../Parachain(1001)
Notes:
What does the location resolve to if evaluated on Parachain(1000)?
---v
Sibling parachain
../Parachain(1001)
---v
Parachain account
Parachain(1000)/AccountId32(0x1234...cdef)
Notes:
What does the location resolve to if evaluated on the relay chain?
---v
Parachain account
Parachain(1000)/AccountId32(0x1234...cdef)
---v
Bridge
../../GlobalConsensus(Kusama)/Parachain(1000)
Notes:
Speak to an example of non-parachain multi-location that would use a bridge XCM reasons about addressing (as in a postal address) that must include understanding where you are, not just where you are going! This will be very powerful later on (Origins)
---v
Bridge
../../GlobalConsensus(Kusama)/Parachain(1000)
Notes:
Even with Bridge Hubs, the relative location is what you'd expect. Bridge Hubs are just a way for routing messages. They are an implementation detail of the transport layer.
---v
Bridge (actual routing)
Notes:
The actual message is routed through Bridge Hub.
Sovereign Accounts
Locations external to the local system can be represented by a local account.
We call this the sovereign account of that location.
They are a mapping from a Location
to an account id.
Notes:
A sovereign account is an account on one system that is controlled by another on a different system. A single account on a system can have multiple sovereign accounts on many other systems. In this example, Alice is an account on AssetHub, and it controls a sovereign account on Collectives.
When transferring between consensus systems, the sovereign account is the one that gets the funds on the destination system.
---v
Sovereign Accounts again
๐ฐ Assets in XCM
Most messages will deal with assets in some way.
How do we address these assets?
---v
Asset Representation
#![allow(unused)] fn main() { struct Asset { pub id: AssetId, pub fun: Fungibility, } struct AssetId(Location); // <- We reuse the location! enum Fungibility { Fungible(u128), NonFungible(AssetInstance), } }
Notes:
We use locations, which we've already discussed, to refer to assets.
A Asset is composed of an asset ID and an enum representing the fungibility of the asset. Asset IDs are the location that leads to the system that issues it, this can be just an index in an assets pallet, for example.
Assets can also either be fungible or non-fungible: Fungible - each token of this asset has the same value as any other NonFungible - each token of this asset is unique and cannot be seen as having the same value as any other token under this asset
---v
Asset filtering and wildcards
#![allow(unused)] fn main() { enum AssetFilter { Definite(Assets), Wild(WildAsset), } enum WildAsset { All, AllOf { id: AssetId, fun: WildFungibility }, // Counted variants } enum WildFungibility { Fungible, NonFungible, } }
Notes:
Sometimes we don't want to specify an asset, but rather filter a collection of them. In this case, we can either list all the assets we want or use a wildcard to select all of them. In reality, it's better to use the counted variant of the wildcards, for benchmarking.
Reanchoring
How do different locations reference the same asset?
Notes:
Locations are relative, so they must be updated and rewritten when sent to another chain, for them to be interpreted correctly.
Native tokens are referenced by the location to their system.
---v
USDT from Asset Hub
PalletInstance(50)/GeneralIndex(1984)
---v
USDT from Bridge Hub
../Parachain(1000)/PalletInstance(50)/GeneralIndex(1984)
---v
Reanchoring to the rescue
๐คน Cross-consensus transfers
Notes:
The two ways of transferring assets between consensus systems are teleports and reserve transfers.
---v
1. Asset teleportation
Notes:
Teleportation works by burning the assets on the source chain and minting them on the destination chain. This method is the simplest one, but requires a lot of trust, since failure to burn or mint on either side will affect the total issuance.
---v
1.1. Example: System parachains?
---v
1.2. Example: Polkadot and Kusama?
---v
2. Reserve asset transfers
Notes:
Reserve asset transfers are more complicated, since they bring in a third actor called the reserve chain. Chain A and B needn't trust each other, they only need to trust the reserve chain. The reserve chain holds the real assets, A and B deal only with derivatives. The transfer is made by burning derivatives from A, moving them from A's SA to B's SA in R, then minting on B.
In some cases, the sender, A, can also be the reserve for a particular asset, in which case the process is simplified, there's no burning of derivatives. This usually happens with parachains' native tokens.
You always trust the issuer of the token to not mint infinite tokens.
---v
2.1. Example: Parachain native tokens
Notes:
Most parachains act as the reserve for their own token. To transfer their token to other chains, they move the real assets to a sovereign account and then tell the chain to mint equivalent derivatives.
---v
2.2. Example: Polkadot to Kusama
Notes:
AssetHub Kusama acts as the reserve for KSM. Kusama doesn't trust Polkadot to teleport KSM to it, but it does trust its own reserve, the AssetHub. Polkadot has a sovereign account in Kusama's AssetHub with some amount of KSM. Whenever some user in Polkadot wants to get KSM on Kusama, they just give the DOT to Polkadot and the KSM are moved from one sovereign account to another. No new trust relationships are added.
Summary
- XCM
- XCM vs XCMP
- Locations
- Sovereign Accounts
- Assets
- Reanchoring
- Cross-consensus transfers
- Teleports
- Reserve asset transfers
---v
Next steps
- Blog series introducing XCM: Parts 1, 2, and 3.
- XCM Format repository
- XCM Docs
---v