tt-metal: [Bounty $500-2500] Bring up microsoft/phi-1 on Wormhole (N150/N300)
Bounty: Enable & Validate microsoft/phi-1 on Tenstorrent Hardware
Description
This bounty involves enabling and validating microsoft/phi-1 on Tenstorrent’s hardware. The goal is to ensure that microsoft/phi-1 compiles, runs end-to-end (inference), meets minimal performance benchmarks, and includes sufficient documentation for community adoption.
Target Model & Difficulty Ratings
- Model:
microsoft/phi-1 - Model Hardware Target: Wormhole (N150 or N300)
- Theoretical Maximum Throughput: 141 t/s/u (tokens per second per user)
Difficulty Ratings (Performance Targets):
- Easy ($500): Achieve ≥ 25% of theoretical maximum throughput (~35 t/s/u).
- Medium ($1500): Achieve ≥ 50% of theoretical maximum throughput (~70 t/s/u).
- Hard ($2500): Achieve ≥ 70% of theoretical maximum throughput (~98 t/s/u).
Success Criteria
1. Functional Bring-Up
microsoft/phi-1compiles and runs on N150/N300 with no blocking errors.
2. Performance Validation
- Clearly document throughput and latency.
- Achieve at least the baseline targets for the selected difficulty rating
3. Accuracy Validation
-
Validate accuracy by running microsoft/phi-1 both on Tenstorrent hardware and a CPU baseline.
-
Document comparative accuracy clearly, meeting or exceeding:
- Achieve top-1 accuracy greater than 80% and top-5 accuracy greater than 95% when compared to CPU baseline.
- Reference current model benchmarks here: Tenstorrent Transformer Performance
4. Documentation
- Clear instructions for building, installing dependencies, and running on Tenstorrent.
- Any relevant parameter tuning or known issues noted.
🔴 Strict (hard requirement, PRs may be rejected)
5. Code Reuse & Clean Structure
- PRs that copy-paste entire codebases or duplicate existing functionality will not be accepted.
- If the model is transformer-based and similar to Llama or Qwen, it must use the tt-transformers base in
tt-metal/models/tt_transformers, reusing modules with minimal changes. - Code should follow a clean, modular structure for internal reuse and customer deliverables.
- Contributors must avoid unnecessary duplication — bring-up is only successful if done in a maintainable, production-ready way.
Bring-up Checklist
- Model functionality
- Post commit CI passes
- Model regression CI testing passes (if applicable)
- Device performance regression CI testing passes (if applicable)
- New/Existing tests provide coverage for changes
- Demo script to show the inference output
Notes
- Work may involve multiple pull requests across relevant repositories (e.g.,
TT-metal, or others). - If additional patches or dependencies are required, please include a short guide for maintainers.
Helpful Links
- We have a model bring up guide that you may use for reference
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 26 (15 by maintainers)
Didn’t get around to this until after devday, but it generated its first token today 😃 Haven’t hit the Easy benchmark yet, will report back.
@cguerreroTT Thanks. I got the machine access and setup is done. Now, I’m able to run tt-metal and I’ll keep posted about the progress.
Currently sitting at 14 t/s/u in collaboration with @aminediro.
Hey @FL33TW00D
Happy to assign you to this.
Be sure that you read over the BOUNTY TERMS, as you agree to all of them by picking up this issue.
If we haven’t received any communication from you in a two week period, we will re-assign the issue, and you will no longer be eligible for completion of this bounty.
If you have any questions feel free to ping me here on this issue! Look forward to seeing your submission.