tt-metal: [Bounty $500-2500] Bring up microsoft/phi-1.5 on Wormhole (N150/N300)

Bounty: Enable & Validate microsoft/phi-1.5 on Tenstorrent Hardware

Description

This bounty involves enabling and validating microsoft/phi-1.5 on Tenstorrent’s hardware. The goal is to ensure that microsoft/phi-1.5 compiles, runs end-to-end (inference), meets minimal performance benchmarks, and includes sufficient documentation for community adoption.

Target Model & Difficulty Ratings

  • Model: microsoft/phi-1.5
  • Model Hardware Target: N150 / N300
  • Theoretical Maximum Throughput: 141 t/s/u (tokens per second per user)

Difficulty Ratings (Performance Targets):

  • Easy ($500): Achieve ≥ 25% of theoretical maximum throughput (~35 t/s/u).
  • Medium ($1500): Achieve ≥ 50% of theoretical maximum throughput (~70 t/s/u).
  • Hard ($2500): Achieve ≥ 70% of theoretical maximum throughput (~98 t/s/u).

Success Criteria

1. Functional Bring-Up

  • microsoft/phi-1.5 compiles and runs on N150/N300 with no blocking errors.

2. Performance Validation

  • Clearly document throughput and latency.
  • Achieve at least the baseline targets for the selected difficulty rating.

3. Accuracy Validation

  • Validate accuracy by running microsoft/phi-1.5 both on Tenstorrent hardware and a CPU baseline.
  • Document comparative accuracy clearly, meeting or exceeding:
    • Top-1 accuracy greater than 80% and top-5 accuracy greater than 95% when compared to CPU baseline.
    • Reference current model benchmarks here: Tenstorrent Transformer Performance

4. Documentation

  • Provide clear instructions for building, installing dependencies, and running on Tenstorrent.
  • Note any relevant parameter tuning or known issues.

🔴 Strict (hard requirement, PRs may be rejected)

5. Code Reuse & Clean Structure

  • PRs that copy-paste entire codebases or duplicate existing functionality will not be accepted.
  • If the model is transformer-based and similar to Llama or Qwen, it must use the tt-transformers base in tt-metal/models/tt_transformers, reusing modules with minimal changes.
  • Code should follow a clean, modular structure for internal reuse and customer deliverables.
  • Contributors must avoid unnecessary duplication — bring-up is only successful if done in a maintainable, production-ready way.

Bring-up Checklist

  • Model functionality verified
  • Post commit CI passes
  • Model regression CI testing passes (if applicable)
  • Device performance regression CI testing passes (if applicable)
  • New/Existing tests provide coverage for changes
  • Demo script to show the inference output

Notes

  • Work may involve multiple pull requests across relevant repositories (e.g., TT-metal or others).
  • If additional patches or dependencies are required, please include a short guide for maintainers.

Helpful Links

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 18 (4 by maintainers)

Most upvoted comments

Hi @cguerreroTT I went to attend the 2025 RISC-V China Summit last week, so I will submit it later. I have successfully compiled tt-metal and modified the module of the phi-1.5 model.

Hey @ajumpa, I have updated the issue description. Please make sure to follow and look at point 5 in the success criteria.

@cguerreroTT University really picked up over the past couple weeks, so progress has been slow. I will update this weekend and I aim to have a WIPR within the next couple weeks.