tt-metal: [Bounty $500-2500] Bring up microsoft/phi-1.5 on Wormhole (N150/N300)
Bounty: Enable & Validate microsoft/phi-1.5 on Tenstorrent Hardware
Description
This bounty involves enabling and validating microsoft/phi-1.5 on Tenstorrent’s hardware. The goal is to ensure that microsoft/phi-1.5 compiles, runs end-to-end (inference), meets minimal performance benchmarks, and includes sufficient documentation for community adoption.
Target Model & Difficulty Ratings
- Model:
microsoft/phi-1.5 - Model Hardware Target: N150 / N300
- Theoretical Maximum Throughput: 141 t/s/u (tokens per second per user)
Difficulty Ratings (Performance Targets):
- Easy ($500): Achieve ≥ 25% of theoretical maximum throughput (~35 t/s/u).
- Medium ($1500): Achieve ≥ 50% of theoretical maximum throughput (~70 t/s/u).
- Hard ($2500): Achieve ≥ 70% of theoretical maximum throughput (~98 t/s/u).
Success Criteria
1. Functional Bring-Up
microsoft/phi-1.5compiles and runs on N150/N300 with no blocking errors.
2. Performance Validation
- Clearly document throughput and latency.
- Achieve at least the baseline targets for the selected difficulty rating.
3. Accuracy Validation
- Validate accuracy by running
microsoft/phi-1.5both on Tenstorrent hardware and a CPU baseline. - Document comparative accuracy clearly, meeting or exceeding:
- Top-1 accuracy greater than 80% and top-5 accuracy greater than 95% when compared to CPU baseline.
- Reference current model benchmarks here: Tenstorrent Transformer Performance
4. Documentation
- Provide clear instructions for building, installing dependencies, and running on Tenstorrent.
- Note any relevant parameter tuning or known issues.
🔴 Strict (hard requirement, PRs may be rejected)
5. Code Reuse & Clean Structure
- PRs that copy-paste entire codebases or duplicate existing functionality will not be accepted.
- If the model is transformer-based and similar to Llama or Qwen, it must use the tt-transformers base in
tt-metal/models/tt_transformers, reusing modules with minimal changes. - Code should follow a clean, modular structure for internal reuse and customer deliverables.
- Contributors must avoid unnecessary duplication — bring-up is only successful if done in a maintainable, production-ready way.
Bring-up Checklist
- Model functionality verified
- Post commit CI passes
- Model regression CI testing passes (if applicable)
- Device performance regression CI testing passes (if applicable)
- New/Existing tests provide coverage for changes
- Demo script to show the inference output
Notes
- Work may involve multiple pull requests across relevant repositories (e.g.,
TT-metalor others). - If additional patches or dependencies are required, please include a short guide for maintainers.
Helpful Links
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 18 (4 by maintainers)
Hi @cguerreroTT I went to attend the 2025 RISC-V China Summit last week, so I will submit it later. I have successfully compiled tt-metal and modified the module of the phi-1.5 model.
Hey @ajumpa, I have updated the issue description. Please make sure to follow and look at point 5 in the success criteria.
@cguerreroTT University really picked up over the past couple weeks, so progress has been slow. I will update this weekend and I aim to have a WIPR within the next couple weeks.