Availability Cores

Deep Dive, Availability Cores

Notes:

Hello!

I'm Bradley Olson

Was student at first Academy

Currently on Parachains Core Team

Will present 3 lectures providing a window into Polkadot core, a slice of where we're at and where we're headed.

First a look at availability cores, the abstraction enabling flexible purchases of blockspace under the umbrella of Polkadot shared security.

Lets get to it

Addressing the Dual Naming

Overview

Review, Blockspace

Blockspace is the capacity of a blockchain
to finalize and commit operations

Polkadot's primary product is blockspace.

Blockspace, Use It or Lose It

Polkadot blockspace is consumed in two ways:

Availability Core Defined

• Availability cores are the abstraction we use to allocate Polkadot's blockspace.

• Allocated via leases and on-demand claims

• Cores divide blockspace supply into discrete units, 1 parachain block per relay chain block per core

• Why "availability"?

• Why "core"?

Notes:

• "Availability", because a core is considered occupied by a parachain block candidate while that candidate is being made available. But cores mediate access to the entire parablock validation pipeline, not just availability.
• "Core", because many candidates can be made available in parallel, mimicking the parallel computation per core in a computer processor.

Availability

Though cores gate the entire parachain block pipeline,
the availability process alone determines when these cores are considered occupied vs free.

To recap, the goals of availability are:

1. To ensure that approvers will always be able to recover the PoVs to validate their assigned blocks

2. To ensure that parachain nodes can recover blocks in the case that a parablock author fails to share them, through malintent or malfunction

Core States

Free

Scheduled

Occupied

Notes:

• Before going any farther we need to talk about core states
• CoreState: Free -> core has not been assigned a parachain via lease or on-demand claim
• CoreState: Scheduled -> Core has an assigned parachain and is currently unoccupied
• CoreState: Occupied -> Core has assignment and is occupied by a parablock pending availability

The Availability Process

Cores and Blockspace Over Time

Notes:

Metaphor:

• Relay chain: Train loading bay
• Relay block: Train leaving station every 6 seconds
• Parachain block: One train car worth of cargo
• Availability core: Car index within all trains

If you have a lease on core 4, then you have the right to fill train car 4 on each train with whatever you want to ship.

Q: How would an on-demand claim be represented in this metaphor?

Mapping Leases and Claims to Validator Subsets

The Big Picture

Notes:

• Which steps of the parachains protocol are missing, and why?
• Going to dive into each piece
• Questions?

Assigning Leases and Claims to Cores

Notes:

• Leases have indices and pair to cores with the same index
• Cores not designated as on-demand and without a paired lease are left free, their blockspace wasted
• When on-demand claims are queued, they are each assigned a designated core in ascending order, looping when reaching the last core

Assigning Backing Groups to Cores

Notes:

• Round robin, fixed intervals

This prevents a byzantine backing group from interrupting the liveness of any one parachain for too long.

Backing Group Formation

Notes:

• Validators randomly assigned to groups at start of session.
• Group count is active validator count / max group size rounded up
• Groups are partitioned such that the largest group and smallest group have a size difference of 1

Assigning Approvers to Cores

Putting the Pieces Together

Occupying Assigned Cores: With Lease

Notes:

Q: What step of the parachains protocol takes place between "Supplied backable candidate" and "availability process?

Occupying Assigned Cores: On Demand

Core States in the Runtime

In file: polkadot/runtime/parachains/src/scheduler.rs

#![allow(unused)]
fn main() {
pub(crate) type AvailabilityCores<T> = StorageValue<_, Vec<Option<CoreOccupied>>, ValueQuery>;

pub enum CoreOccupied {
    /// A parathread (on-demand parachain).
    Parathread(ParathreadEntry),
    /// A lease holding parachain.
    Parachain,
}

/// Parathread = on-demand parachain
pub struct ParathreadEntry {
	/// The claim.
	pub claim: ParathreadClaim,
	/// Number of retries.
	pub retries: u32,
}
}

Notes:

Q: When Option is None, what does that indicate?

• Which para occupies a core is stored separately in the following structure

Core Assignments in The Runtime

In file: polkadot/runtime/parachains/src/scheduler.rs

#![allow(unused)]
fn main() {
pub(crate) type Scheduled<T> = StorageValue<_, Vec<CoreAssignment>, ValueQuery>;

pub struct CoreAssignment {
    /// The core that is assigned.
    pub core: CoreIndex,
    /// The unique ID of the para that is assigned to the core.
    pub para_id: ParaId,
    /// The kind of the assignment.
    pub kind: AssignmentKind,
}

pub enum AssignmentKind {
	/// A parachain.
	Parachain,
	/// A parathread (on-demand parachain).
	Parathread(CollatorId, u32),
}
}

Notes:

• Vec of all core assignments
• Pairs ParaId with CoreIndex
• AssignmentKind carries retry info for on-demand

How Cores Gate Each Step of the Parachains Protocol

How Core Assignments Mediate Backing

Each parablock candidate is built in the context of a particular relay_parent.

Validators query their core assignment as of relay_parent and refuse to second candidates not associated with their backing group.

Notes:

• Relay parent context: max PoV size, current parachain runtime code, and backing group assignments.

How Core Assignments Mediate Backing (Cont.)

handle_second_message() in the Backing Subsystem.

#![allow(unused)]
fn main() {
if Some(candidate.descriptor().para_id) != rp_state.assignment {
	gum::debug!(
		target: LOG_TARGET,
		our_assignment = ?rp_state.assignment,
		collation = ?candidate.descriptor().para_id,
		"Subsystem asked to second for para outside of our assignment",
	);

	return Ok(())
}
}

Cores and Backing On-Chain

Notes:

• Review possible core states
• Mention time-out vs made available

Q: What does "immediately occupies core" imply?

Cores and Backing On-Chain, Code

#![allow(unused)]
fn main() {
	let (para_id) = match core {
		CoreState::Scheduled(scheduled_core) => {
			scheduled_core.para_id
		},
		CoreState::Occupied(occupied_core) => {
			if current_occupant_made_available(core_idx, &occupied_core.availability)
			{
				if let Some(ref scheduled_core) = occupied_core.next_up_on_available {
					scheduled_core.para_id
				} else {
					continue
				}
			} else {
				if occupied_core.time_out_at != current_block {
					continue
				}
				if let Some(ref scheduled_core) = occupied_core.next_up_on_time_out {
					scheduled_core.para_id
				} else {
					continue
				}
			}
		},
		CoreState::Free => continue,
	};

	let candidate_hash =
		get_backable_candidate(relay_parent, para_id, required_path, sender).await?;
}

Notes:

• Discuss core freeing criteria
• bitfields_indicate_availability
  • next_up_on_available
• availability time out
  • next_up_on_timeout

Cores and Approvals, Disputes, Finality

Each included candidate has already occupied an availability core

Approvals, Disputes, and Finality are only provided to included candidates

Advantages Cores Give us Now

Notes:

• Regularly rotating pairings between cores and backing groups lessen the impact of potential bad validator subsets
• Cores accommodate advance allocation making costs for parachains predictable
• Core time can be resold or split for precise allocation

Cores and Roadmap Items

Exotic core scheduling

Notes:

• Each color represents a parachain Metaphor:
• Multiple cars on the same train
• Overlapping time spans of rights to fill different train cars
• Long term rights to one car and buy rights to others for just one train

Divisible and Marketable Blockspace

We want Parachains to buy, sell, and split blockspace such that they are allocated exactly as much as fits their needs.

Notes:

• Example: Region spanning 100 blocks. Split the region use so that each of two parties can submit up to 50 parablocks. Ownership proportion is enforced throughout the region such that each party can't submit more than 5 out of the first 10 blocks.

Framing Shift: Blockspace vs Core Time

Blockspace is a universal term for the product of blockchains, while core time is Polkadot's particular abstraction for allocating blockspace or other computation.

Cores can secure computation other than blocks. For example, a smart contract could be deployed on a core directly.

Allocating cores by time rather than for a fixed block size could allow for smaller or larger parachain blocks to be handled seamlessly.

Resources

Questions