tt-metal: [Bounty $500-2500] Bring up microsoft/phi-4 on Wormhole (N300/LB/QB)

Bounty: Enable & Validate microsoft/phi-4 on Tenstorrent Hardware

Description

This bounty involves enabling and validating microsoft/phi-4 on Tenstorrent’s hardware. The goal is to ensure that microsoft/phi-4 compiles, runs end-to-end (inference), meets minimal performance benchmarks, and includes sufficient documentation for community adoption.

Target Model & Difficulty Ratings

  • Model: microsoft/phi-4
  • Model Hardware Target: Wormhole (N300/QB/LB)
  • Theoretical Maximum Throughput: 33 t/s/u (tokens per second per user)

Difficulty Ratings (Performance Targets):

  • Easy ($500): Achieve ≥ 25% of theoretical maximum throughput (~8 t/s/u).
  • Medium ($1500): Achieve ≥ 50% of theoretical maximum throughput (~16 t/s/u).
  • Hard ($2500): Achieve ≥ 70% of theoretical maximum throughput (~23 t/s/u).

Success Criteria

1. Functional Bring-Up

  • microsoft/phi-4 compiles and runs on N300/QB/LB with no blocking errors.

2. Performance Validation

  • Clearly document throughput and latency.
  • Achieve at least the baseline targets for the selected difficulty rating

3. Accuracy Validation

  • Validate accuracy by running microsoft/phi-4 both on Tenstorrent hardware and a CPU baseline.

  • Document comparative accuracy clearly, meeting or exceeding:

    • Achieve top-1 accuracy greater than 80% and top-5 accuracy greater than 95% when compared to CPU baseline.
    • Reference current model benchmarks here: Tenstorrent Transformer Performance

3. Documentation

  • Clear instructions for building, installing dependencies, and running on Tenstorrent.
  • Any relevant parameter tuning or known issues noted.

Bring-up Checklist

  • Model functionality
  • Post commit CI passes
  • Model regression CI testing passes (if applicable)
  • Device performance regression CI testing passes (if applicable)
  • New/Existing tests provide coverage for changes
  • Demo script to show the inference output

Notes

  • Work may involve multiple pull requests across relevant repositories (e.g., TT-metal, or others).
  • If additional patches or dependencies are required, please include a short guide for maintainers.

Helpful Links

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 18 (2 by maintainers)

Most upvoted comments

Hi @AndrewZhaoLuo, since we never heard back from @MirMustafaAli, I’ve assigned the issue to you.

Be sure that you read over the BOUNTY TERMS, as you agree to all of them by picking up this issue.

If we haven’t received any communication from you in a two week period, we will re-assign the issue, and you will no longer be eligible for completion of this bounty.

If you have any questions feel free to ping someone here on this issue. Look forward to seeing your submission!

Hey @Shubhamsaboo Quick update, I received access to N300 on Koyeb today. Thanks for the help!