grail - incentivising intelligence

Overview

grail focuses on post-training language models with verifiable inference capabilities. It leverages the GRAIL protocol (Guaranteed Rollout Authenticity via Inference Ledger) to ensure cryptographically verifiable model outputs during reinforcement learning rollouts.

Key Distinction: grail vs GRAIL

grail (lowercase): The Bittensor subnet implementation for post-training models using verifiable inference
GRAIL (uppercase): The underlying cryptographic protocol (Guaranteed Rollout Authenticity via Inference Ledger) that provides verifiable inference guarantees

Architecture

Core Components

1. GRAIL Protocol (`grail/grail.py`)

The cryptographic protocol implementation that ensures verifiable inference through:

Pseudorandom function (PRF) for deterministic token generation
Sketch-based proof system for model output verification
Integration with drand beacon for randomness
Support for SAT problem generation and verification

2. Rollout Generation (`grail/rollout.py`)

GRPO (Generalized Proximal Policy Optimization) rollout system for:

Multiple rollouts per problem for robust training
Token-level logprob tracking for GRAIL proofs
Advantage computation for policy gradient training
Trajectory recording for analysis

3. Environment System (`grail/environments/`)

Modular environment framework supporting:

SAT Problems (sat.py): 3-SAT problem generation and solving
Extensible base for additional problem domains
Text-based problem representation for LLM processing

4. Communication Layer (`grail/comms.py`)

Handles subnet communication and coordination between validators and miners.

5. Randomness (`grail/drand.py`)

Integration with drand beacon for verifiable randomness in problem generation.

How It Works

Post-Training Flow

Problem Generation: Validators generate verifiable problems (e.g., SAT instances) using drand beacon randomness
Rollout Collection: Miners generate multiple solution attempts using their models
GRAIL Verification: Each rollout is cryptographically verified using the GRAIL protocol
Reward Computation: Valid rollouts receive rewards based on solution quality
Model Updates: Miners update their models using GRPO with collected rollouts

Verifiable Inference

The GRAIL protocol ensures that:

Model outputs are deterministically verifiable
Rollouts cannot be falsified or replayed
Randomness is publicly verifiable through drand
Token generation follows cryptographic constraints

Technical Details

GRAIL Protocol Parameters

PRIME_Q: 2,147,483,647 (modular arithmetic prime)
CHALLENGE_K: 16 (number of challenge tokens)
TOLERANCE: 3 (allowed deviations in verification)

Supported Environments

SAT Problems: 3-SAT instances with configurable complexity
- Variables: 3-10
- Clauses: 5-20
- Clause length: 3 (3-SAT)

Model Requirements

Compatible with HuggingFace transformers
Support for logprob extraction
CUDA-enabled for efficient inference