litex-boards: Alveo U280 DDR4 problem

Hello all,

I am working on adding support for the Alveo U280 board (in my case it is ES1) and am using the Alveo U250 platform and target as reference. I was able to generate a minimal SoC and bitstream and boot LiteX, however memory initialization fails. Included below is the boot log including a failing memory test. The board files are at my fork under alveo_u280 branch.

The Alveo U280 is similar to the U250, with a few differences. It has 2 (not 4) DDR4 channels and HBM2. The main clock sources are of 100MHz (not 300MHz). The same MTA18ASF2G72PZ DDR4 module is used. I verified the configuration settings of two identical Vivado MIG+MicroBlaze default designs for the two boards and only the CLK and pins are different.

Any idea what I am doing wrong? Any suggestions are greatly appreciated. Thank you!

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2020 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS built on Jan 26 2021 21:29:42
 BIOS CRC passed (125b79b6)

 Migen git sha1: --------
 LiteX git sha1: 737ed9d6

--=============== SoC ==================--
CPU:            VexRiscv SMP-LINUX @ 125MHz
BUS:            WISHBONE 32-bit @ 4GiB
CSR:            32-bit data
ROM:            64KiB
SRAM:           8KiB
L2:             0KiB
SDRAM:          1048576KiB 64-bit @ 1000MT/s (CL-9 CWL-9)

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Write leveling:
  Cmd/Clk scan (0-324)
  |000000  |0000  |0000  |0000| best: -1
  Setting Cmd/Clk delay to -1 taps.
  Data scan:
  m0: |1111111111111111111111| delay: -
  m1: |1111111111111111111111| delay: -
  m2: |1111111111111111111111| delay: -
  m3: |1111111111111111111111| delay: -
  m4: |1111111111111111111111| delay: -
  m5: |1111111111111111111111| delay: -
  m6: |1111111111111111111111| delay: -
  m7: |1111111111111111111111| delay: -
Write latency calibration:
m0:0 m1:0 m2:0 m3:0 m4:0 m5:0 m6:0 m7:0
Read leveling:
  m0, b0: |00000000000000000000000000000000| delays: -
  m0, b1: |00000000000000000000000000000000| delays: -
  m0, b2: |00000000000000000000000000000000| delays: -
  m0, b3: |00000000000000000000000000000000| delays: -
  m0, b4: |00000000000000000000000000000000| delays: -
  m0, b5: |00000000000000000000000000000000| delays: -
  m0, b6: |00000000000000000000000000000000| delays: -
  m0, b7: |00000000000000000000000000000000| delays: -
  best: m0, b00 delays: -
  m1, b0: |00000000000000000000000000000000| delays: -
  m1, b1: |00000000000000000000000000000000| delays: -
  m1, b2: |00000000000000000000000000000000| delays: -
  m1, b3: |00000000000000000000000000000000| delays: -
  m1, b4: |00000000000000000000000000000000| delays: -
  m1, b5: |00000000000000000000000000000000| delays: -
  m1, b6: |00000000000000000000000000000000| delays: -
  m1, b7: |00000000000000000000000000000000| delays: -
  best: m1, b00 delays: -
  m2, b0: |00000000000000000000000000000000| delays: -
  m2, b1: |00000000000000000000000000000000| delays: -
  m2, b2: |00000000000000000000000000000000| delays: -
  m2, b3: |00000000000000000000000000000000| delays: -
  m2, b4: |00000000000000000000000000000000| delays: -
  m2, b5: |00000000000000000000000000000000| delays: -
  m2, b6: |00000000000000000000000000000000| delays: -
  m2, b7: |00000000000000000000000000000000| delays: -
  best: m2, b00 delays: -
  m3, b0: |00000000000000000000000000000000| delays: -
  m3, b1: |00000000000000000000000000000000| delays: -
  m3, b2: |00000000000000000000000000000000| delays: -
  m3, b3: |00000000000000000000000000000000| delays: -
  m3, b4: |00000000000000000000000000000000| delays: -
  m3, b5: |00000000000000000000000000000000| delays: -
  m3, b6: |00000000000000000000000000000000| delays: -
  m3, b7: |00000000000000000000000000000000| delays: -
  best: m3, b00 delays: -
  m4, b0: |00000000000000000000000000000000| delays: -
  m4, b1: |00000000000000000000000000000000| delays: -
  m4, b2: |00000000000000000000000000000000| delays: -
  m4, b3: |00000000000000000000000000000000| delays: -
  m4, b4: |00000000000000000000000000000000| delays: -
  m4, b5: |00000000000000000000000000000000| delays: -
  m4, b6: |00000000000000000000000000000000| delays: -
  m4, b7: |00000000000000000000000000000000| delays: -
  best: m4, b00 delays: -
  m5, b0: |00000000000000000000000000000000| delays: -
  m5, b1: |00000000000000000000000000000000| delays: -
  m5, b2: |00000000000000000000000000000000| delays: -
  m5, b3: |00000000000000000000000000000000| delays: -
  m5, b4: |00000000000000000000000000000000| delays: -
  m5, b5: |00000000000000000000000000000000| delays: -
  m5, b6: |00000000000000000000000000000000| delays: -
  m5, b7: |00000000000000000000000000000000| delays: -
  best: m5, b00 delays: -
  m6, b0: |00000000000000000000000000000000| delays: -
  m6, b1: |00000000000000000000000000000000| delays: -
  m6, b2: |00000000000000000000000000000000| delays: -
  m6, b3: |00000000000000000000000000000000| delays: -
  m6, b4: |00000000000000000000000000000000| delays: -
  m6, b5: |00000000000000000000000000000000| delays: -
  m6, b6: |00000000000000000000000000000000| delays: -
  m6, b7: |00000000000000000000000000000000| delays: -
  best: m6, b00 delays: -
  m7, b0: |00000000000000000000000000000000| delays: -
  m7, b1: |00000000000000000000000000000000| delays: -
  m7, b2: |00000000000000000000000000000000| delays: -
  m7, b3: |00000000000000000000000000000000| delays: -
  m7, b4: |00000000000000000000000000000000| delays: -
  m7, b5: |00000000000000000000000000000000| delays: -
  m7, b6: |00000000000000000000000000000000| delays: -
  m7, b7: |00000000000000000000000000000000| delays: -
  best: m7, b00 delays: -
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2MiB)...
  Write: 0x40000000-0x40200000 2MiB     
   Read: 0x40000000-0x40200000 2MiB     
  bus errors:  256/256
  addr errors: 8192/8192
  data errors: 524288/524288
Memtest KO
Memory initialization failed

--============= Console ================--

litex> mem_list

Available memory regions:
PLIC      0xf0c00000 0x400000 
CLINT     0xf0010000 0x10000 
ROM       0x00000000 0x10000 
SRAM      0x10000000 0x2000 
MAIN_RAM  0x40000000 0x40000000 
FIRMWARE_RAM  0x20000000 0x8000 
OPENSBI   0x40f00000 0x80000 
CSR       0xf0000000 0x10000 

litex> mem_test 0x40000000 1024

Memtest at 0x40000000 (1KiB)...
  Write: 0x40000000-0x40000400 1KiB   
   Read: 0x40000000-0x40000400 1KiB   
  bus errors:  256/256
  addr errors: 256/256
  data errors: 256/256
Memtest KO

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (15 by maintainers)

Most upvoted comments

Thanks @smosanu for the update. 300MHz sys_clk is a bit high for now and LiteDRAM hasn’t been tested at this speed yet. I would recommend using 150MHz sys_clk to start with and then increase progressively when working.

+1
enjoy-digital on Mar 24, 2021

Thanks @smosanu, that’s encouraging. The last issue could be related to wrong electrical settings. Using ddr4_mr_gen (https://github.com/enjoy-digital/litedram/blob/master/bench/ddr4_mr_gen.py) could allow you to do some tests with various settings without re-compiling the bistream. I’ll try to give more indications on this soon.

+1
enjoy-digital on Feb 9, 2021

That’s already better and you have first sign of life on m5. The calibration still requires some initial adjustments and will requires some tests:

Can you try to do the following sequence in the BIOS:

sdram_force_cmd_delay N
sdram_cal
sdram_test

With N in from 0 to 512 with steps of ~50 and share results.

If no improvements, could you try with cmd_latency set to 1 here: https://github.com/enjoy-digital/litedram/blob/master/litedram/phy/usddrphy.py#L33? And first see with the automatic calibration, then if not working doing the same sdram_force_cmd_delay test than the one done with the default cmd_latency?

+1
enjoy-digital on Jan 29, 2021

Hi @smosanu,

thanks for the log. This behavior generally means there are pinout errors, constraint errors or missing INTERNAL_VREF constraints. Setting INTERNAL_VREF to 0.84 on the banks used by the LiteDRAM controller is mandatory to get the DDR4 working and I don’t see the constraints in your platform file. For an example you can have a look at: https://github.com/litex-hub/litex-boards/blob/master/litex_boards/platforms/alveo_u250.py#L351-L366

Once added, could you provide an updated log if still failing?

+1
enjoy-digital on Jan 27, 2021
Read more comments on GitHub
← linux-on-litex-vexriscv: SDRAM initialization fails on some OrangeCrabs when LDM/UDM are used independently
lithops: Cannot run Hello World against Azure →