tt-metal: [Bounty $500-2500] Bring up microsoft/phi-2 on Wormhole (N150/N300)
Bounty: Enable & Validate microsoft/phi-2 on Tenstorrent Hardware
Description
This bounty involves enabling and validating microsoft/phi-2 on Tenstorrent’s hardware. The goal is to ensure that microsoft/phi-2 compiles, runs end-to-end (inference), meets minimal performance benchmarks, and includes sufficient documentation for community adoption.
Target Model & Difficulty Ratings
- Model:
microsoft/phi-2 - Model Hardware Target: N150 / N300
- Theoretical Maximum Throughput: 68 t/s/u (tokens per second per user)
Difficulty Ratings (Performance Targets):
- Easy ($500): Achieve ≥ 25% of theoretical maximum throughput (~17 t/s/u).
- Medium ($1500): Achieve ≥ 50% of theoretical maximum throughput (~34 t/s/u).
- Hard ($2500): Achieve ≥ 70% of theoretical maximum throughput (~48 t/s/u).
Success Criteria
1. Functional Bring-Up
microsoft/phi-2compiles and runs on N150/N300/LoudBox with no blocking errors.
2. Performance Validation
- Clearly document throughput and latency.
- Achieve at least the baseline targets for the selected difficulty rating.
3. Accuracy Validation
- Validate accuracy by running
microsoft/phi-2both on Tenstorrent hardware and a CPU baseline. - Document comparative accuracy clearly, meeting or exceeding:
- Top-1 accuracy greater than 80% and top-5 accuracy greater than 95% when compared to CPU baseline.
- Reference current model benchmarks here: Tenstorrent Transformer Performance
4. Documentation
- Provide clear instructions for building, installing dependencies, and running on Tenstorrent.
- Note any relevant parameter tuning or known issues.
Bring-up Checklist
- Model functionality verified
- Post commit CI passes
- Model regression CI testing passes (if applicable)
- Device performance regression CI testing passes (if applicable)
- New/Existing tests provide coverage for changes
- Demo script to show the inference output
Notes
- Work may involve multiple pull requests across relevant repositories (e.g.,
TT-metalor others). - If additional patches or dependencies are required, please include a short guide for maintainers.
Helpful Links
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 34 (13 by maintainers)
benrawane2012@gmail.com
Hello again, just to keep you posted about the progress, I’ve small issue concern decoding the output/resizing the tiles
@Shubhamsaboo I’m on it, I’ll submit next days ASAP, thanks
The expected deliverable will be a table like the reference table (accuracy plus performance) plus a video recording of of how to install and compile the model plus how to running the inference