Table of interleaving 2-4 controllers ===================================== +--------------+-----------------------------------------------------------+ |Configuration | Memory Controller | | | 1 2 3 4 | |--------------+--------------+--------------+-----------------------------+ | Two memory | Not Intlv'ed | Not Intlv'ed | | | complexes +--------------+--------------+ | | | 2-way Intlv'ed | | |--------------+--------------+--------------+--------------+ | | | Not Intlv'ed | Not Intlv'ed | Not Intlv'ed | | | Three memory +--------------+--------------+--------------+ | | complexes | 2-way Intlv'ed | Not Intlv'ed | | | +-----------------------------+--------------+ | | | 3-way Intlv'ed | | +--------------+--------------+--------------+--------------+--------------+ | | Not Intlv'ed | Not Intlv'ed | Not Intlv'ed | Not Intlv'ed | | Four memory +--------------+--------------+--------------+--------------+ | complexes | 2-way Intlv'ed | 2-way Intlv'ed | | +-----------------------------+-----------------------------+ | | 4-way Intlv'ed | +--------------+-----------------------------------------------------------+ Table of 2-way interleaving modes supported in cpu/8xxx/ddr/ ====================================================== +-------------+---------------------------------------------------------+ | | Rank Interleaving | | +--------+-----------+-----------+------------+-----------+ |Memory | | | | 2x2 | 4x1 | |Controller | None | 2x1 lower | 2x1 upper | {CS0+CS1}, | {CS0+CS1+ | |Interleaving | | {CS0+CS1} | {CS2+CS3} | {CS2+CS3} | CS2+CS3} | +-------------+--------+-----------+-----------+------------+-----------+ |None | Yes | Yes | Yes | Yes | Yes | +-------------+--------+-----------+-----------+------------+-----------+ |Cacheline | Yes | Yes | No | No, Only(*)| Yes | | |CS0 Only| | | {CS0+CS1} | | +-------------+--------+-----------+-----------+------------+-----------+ |Page | Yes | Yes | No | No, Only(*)| Yes | | |CS0 Only| | | {CS0+CS1} | | +-------------+--------+-----------+-----------+------------+-----------+ |Bank | Yes | Yes | No | No, Only(*)| Yes | | |CS0 Only| | | {CS0+CS1} | | +-------------+--------+-----------+-----------+------------+-----------+ |Superbank | No | Yes | No | No, Only(*)| Yes | | | | | | {CS0+CS1} | | +-------------+--------+-----------+-----------+------------+-----------+ (*) Although the hardware can be configured with memory controller interleaving using "2x2" rank interleaving, it only interleaves {CS0+CS1} from each controller. {CS2+CS3} on each controller are only rank interleaved on that controller. For memory controller interleaving, identical DIMMs are suggested. Software doesn't check the size or organization of interleaved DIMMs. The ways to configure the ddr interleaving mode ============================================== 1. In board header file(e.g.MPC8572DS.h), add default interleaving setting under "CONFIG_EXTRA_ENV_SETTINGS", like: #define CONFIG_EXTRA_ENV_SETTINGS \ "hwconfig=fsl_ddr:ctlr_intlv=bank" \ ...... 2. Run U-Boot "setenv" command to configure the memory interleaving mode. Either numerical or string value is accepted. # disable memory controller interleaving setenv hwconfig "fsl_ddr:ctlr_intlv=null" # cacheline interleaving setenv hwconfig "fsl_ddr:ctlr_intlv=cacheline" # page interleaving setenv hwconfig "fsl_ddr:ctlr_intlv=page" # bank interleaving setenv hwconfig "fsl_ddr:ctlr_intlv=bank" # superbank setenv hwconfig "fsl_ddr:ctlr_intlv=superbank" # 1KB 3-way interleaving setenv hwconfig "fsl_ddr:ctlr_intlv=3way_1KB" # 4KB 3-way interleaving setenv hwconfig "fsl_ddr:ctlr_intlv=3way_4KB" # 8KB 3-way interleaving setenv hwconfig "fsl_ddr:ctlr_intlv=3way_8KB" # disable bank (chip-select) interleaving setenv hwconfig "fsl_ddr:bank_intlv=null" # bank(chip-select) interleaving cs0+cs1 setenv hwconfig "fsl_ddr:bank_intlv=cs0_cs1" # bank(chip-select) interleaving cs2+cs3 setenv hwconfig "fsl_ddr:bank_intlv=cs2_cs3" # bank(chip-select) interleaving (cs0+cs1) and (cs2+cs3) (2x2) setenv hwconfig "fsl_ddr:bank_intlv=cs0_cs1_and_cs2_cs3" # bank(chip-select) interleaving (cs0+cs1+cs2+cs3) (4x1) setenv hwconfig "fsl_ddr:bank_intlv=cs0_cs1_cs2_cs3" # bank(chip-select) interleaving (auto) setenv hwconfig "fsl_ddr:bank_intlv=auto" This auto mode only select from cs0_cs1_cs2_cs3, cs0_cs1, null dependings on DIMMs. Memory controller address hashing ================================== If the DDR controller supports address hashing, it can be enabled by hwconfig. Syntax is: hwconfig=fsl_ddr:addr_hash=true Memory controller ECC on/off ============================ If ECC is enabled in board configuratoin file, i.e. #define CONFIG_DDR_ECC, ECC can be turned on/off by hwconfig. Syntax is hwconfig=fsl_ddr:ecc=off Memory address parity on/off ============================ address parity can be turned on/off by hwconfig. Syntax is: hwconfig=fsl_ddr:parity=on Memory testing options for mpc85xx ================================== 1. Memory test can be done once U-Boot prompt comes up using mtest, or 2. Memory test can be done with Power-On-Self-Test function, activated at compile time. In order to enable the POST memory test, CFG_POST needs to be defined in board configuraiton header file. By default, POST memory test performs a fast test. A slow test can be enabled by changing the flag at compiling time. To test memory bigger than 2GB, 36BIT support is needed. Memory is tested within a 2GB window. TLBs are used to map the virtual 2GB window to physical address so that all physical memory can be tested. Combination of hwconfig ======================= Hwconfig can be combined with multiple parameters, for example, on a supported platform hwconfig=fsl_ddr:addr_hash=true,ctlr_intlv=cacheline,bank_intlv=cs0_cs1_cs2_cs3,ecc=on Table for dynamic ODT for DDR3 ============================== For single-slot system with quad-rank DIMM and dual-slot system, dynamic ODT may be needed, depending on the configuration. The numbers in the following tables are in Ohms. * denotes dynamic ODT Two slots system +-----------------------+----------+---------------+-----------------------------+-----------------------------+ | Configuration | |DRAM controller| Slot 1 | Slot 2 | +-----------+-----------+----------+-------+-------+--------------+--------------+--------------+--------------+ | | | | | | Rank 1 | Rank 2 | Rank 1 | Rank 2 | + Slot 1 | Slot 2 |Write/Read| Write | Read |-------+------+-------+------+-------+------+-------+------+ | | | | | | Write | Read | Write | Read | Write | Read | Write | Read | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 1 | off | 75 | 120 | off | off | off | off | off | 30 | 30 | | Dual Rank | Dual Rank |----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 2 | off | 75 | off | off | 30 | 30 | 120 | off | off | off | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 1 | off | 75 | 120 | off | off | off | 20 | 20 | | | | Dual Rank |Single Rank|----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 2 | off | 75 | off | off | 20 | 20 | 120 *| off | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 1 | off | 75 | 120 *| off | | | off | off | 20 | 20 | |Single Rank| Dual Rank |----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 2 | off | 75 | 20 | 20 | | | 120 | off | off | off | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 1 | off | 75 | 120 *| off | | | 30 | 30 | | | |Single Rank|Single Rank|----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 2 | off | 75 | 30 | 30 | | | 120 *| off | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | Dual Rank | Empty | Slot 1 | off | 75 | 40 | off | off | off | | | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | Empty | Dual Rank | Slot 2 | off | 75 | | | | | 40 | off | off | off | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ |Single Rank| Empty | Slot 1 | off | 75 | 40 | off | | | | | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | Empty |Single Rank| Slot 2 | off | 75 | | | | | 40 | off | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ Single slot system +-------------+------------+---------------+-----------------------------+-----------------------------+ | | |DRAM controller| Rank 1 | Rank 2 | Rank 3 | Rank 4 | |Configuration| Write/Read |-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Write | Read | Write | Read | Write | Read | Write | Read | Write | Read | +-------------+------------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | R1 | off | 75 | 120 *| off | off | off | 20 | 20 | off | off | | |------------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | R2 | off | 75 | off | 20 | 120 | off | 20 | 20 | off | off | | Quad Rank |------------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | R3 | off | 75 | 20 | 20 | off | off | 120 *| off | off | off | | |------------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | R4 | off | 75 | 20 | 20 | off | off | off | 20 | 120 | off | +-------------+------------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | R1 | off | 75 | 40 | off | off | off | | Dual Rank |------------+-------+-------+-------+------+-------+------+ | | R2 | off | 75 | 40 | off | off | off | +-------------+------------+-------+-------+-------+------+-------+------+ | Single Rank | R1 | off | 75 | 40 | off | +-------------+------------+-------+-------+-------+------+ Reference http://www.xrosstalkmag.com/mag_issues/xrosstalk_oct08_final.pdf http://download.micron.com/pdf/technotes/ddr3/tn4108_ddr3_design_guide.pdf Table for ODT for DDR2 ====================== Two slots system +-----------------------+----------+---------------+-----------------------------+-----------------------------+ | Configuration | |DRAM controller| Slot 1 | Slot 2 | +-----------+-----------+----------+-------+-------+--------------+--------------+--------------+--------------+ | | | | | | Rank 1 | Rank 2 | Rank 1 | Rank 2 | + Slot 1 | Slot 2 |Write/Read| Write | Read |-------+------+-------+------+-------+------+-------+------+ | | | | | | Write | Read | Write | Read | Write | Read | Write | Read | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 1 | off | 150 | off | off | off | off | 75 | 75 | off | off | | Dual Rank | Dual Rank |----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 2 | off | 150 | 75 | 75 | off | off | off | off | off | off | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 1 | off | 150 | off | off | off | off | 75 | 75 | | | | Dual Rank |Single Rank|----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 2 | off | 150 | 75 | 75 | off | off | off | off | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 1 | off | 150 | off | off | | | 75 | 75 | off | off | |Single Rank| Dual Rank |----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 2 | off | 150 | 75 | 75 | | | off | off | off | off | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 1 | off | 150 | off | off | | | 75 | 75 | | | |Single Rank|Single Rank|----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | | | Slot 2 | off | 150 | 75 | 75 | | | off | off | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | Dual Rank | Empty | Slot 1 | off | 75 | 150 | off | off | off | | | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | Empty | Dual Rank | Slot 2 | off | 75 | | | | | 150 | off | off | off | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ |Single Rank| Empty | Slot 1 | off | 75 | 150 | off | | | | | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ | Empty |Single Rank| Slot 2 | off | 75 | | | | | 150 | off | | | +-----------+-----------+----------+-------+-------+-------+------+-------+------+-------+------+-------+------+ Single slot system +-------------+------------+---------------+-----------------------------+ | | |DRAM controller| Rank 1 | Rank 2 | |Configuration| Write/Read |-------+-------+-------+------+-------+------+ | | | Write | Read | Write | Read | Write | Read | +-------------+------------+-------+-------+-------+------+-------+------+ | | R1 | off | 75 | 150 | off | off | off | | Dual Rank |------------+-------+-------+-------+------+-------+------+ | | R2 | off | 75 | 150 | off | off | off | +-------------+------------+-------+-------+-------+------+-------+------+ | Single Rank | R1 | off | 75 | 150 | off | +-------------+------------+-------+-------+-------+------+ Reference http://www.samsung.com/global/business/semiconductor/products/dram/downloads/applicationnote/ddr2_odt_control_200603.pdf Interactive DDR debugging =========================== For DDR parameter tuning up and debugging, the interactive DDR debugger can be activated by setting the environment variable "ddr_interactive" to any value. (The value of ddr_interactive may have a meaning in the future, but, for now, the presence of the variable will cause the debugger to run.) Once activated, U-Boot will show the prompt "FSL DDR>" before enabling the DDR controller. The available commands are printed by typing "help". Another way to enter the interactive DDR debugger without setting the environment variable is to send the 'd' character early during the boot process. To save booting time, no additional delay is added, so the window to send the key press is very short -- basically, it is the time before the memory controller code starts to run. For example, when rebooting from within U-Boot, the user must press 'd' IMMEDIATELY after hitting enter to initiate a 'reset' command. In case of power on/reset, the user can hold down the 'd' key while applying power or hitting the board's reset button. The example flow of using interactive debugging is type command "compute" to calculate the parameters from the default type command "print" with arguments to show SPD, options, registers type command "edit" with arguments to change any if desired type command "copy" with arguments to copy controller/dimm settings type command "go" to continue calculation and enable DDR controller Additional commands to restart the debugging are: type command "reset" to reset the board type command "recompute" to reload SPD and start over Note, check "next_step" to show the flow. For example, after edit opts, the next_step is STEP_ASSIGN_ADDRESSES. After editing registers, the next_step is STEP_PROGRAM_REGS. Upon issuing command "go", the debugger will program the DDR controller with the current setting without further calculation and then exit to resume the booting of the machine. The detail syntax for each commands are print [c] [d] [spd] [dimmparms] [commonparms] [opts] [addresses] [regs] c - the controller number, eg. c0, c1 d - the DIMM number, eg. d0, d1 spd - print SPD data dimmparms - DIMM parameters, calculated from SPD commonparms - lowest common parameters for all DIMMs opts - options addresses - address assignment (not implemented yet) regs - controller registers edit c - the controller number, eg. c0, c1 d - the DIMM number, eg. d0, d1 spd - print SPD data dimmparms - DIMM parameters, calculated from SPD commonparms - lowest common parameters for all DIMMs opts - options addresses - address assignment (not implemented yet) regs - controller registers - name of the modified element byte number if the object is SPD - decimal or heximal (prefixed with 0x) numbers copy same as for "edit" command DIMM numbers ignored for commonparms, opts, and regs reset no arguement - reset the board recompute no argument - reload SPD and start over compute no argument - recompute from current next_step next_step no argument - show current next_step help no argument - print a list of all commands go no argument - program memory controller(s) and continue with U-Boot Examples of debugging flow FSL DDR>compute Detected UDIMM UG51U6400N8SU-ACF FSL DDR>print print [c] [d] [spd] [dimmparms] [commonparms] [opts] [addresses] [regs] FSL DDR>print dimmparms DIMM parameters: Controller=0 DIMM=0 DIMM organization parameters: module part name = UG51U6400N8SU-ACF rank_density = 2147483648 bytes (2048 megabytes) capacity = 4294967296 bytes (4096 megabytes) burst_lengths_bitmask = 0C base_addresss = 0 (00000000 00000000) n_ranks = 2 data_width = 64 primary_sdram_width = 64 ec_sdram_width = 0 registered_dimm = 0 n_row_addr = 15 n_col_addr = 10 edc_config = 0 n_banks_per_sdram_device = 8 tCKmin_X_ps = 1500 tCKmin_X_minus_1_ps = 0 tCKmin_X_minus_2_ps = 0 tCKmax_ps = 0 caslat_X = 960 tAA_ps = 13125 caslat_X_minus_1 = 0 caslat_X_minus_2 = 0 caslat_lowest_derated = 0 tRCD_ps = 13125 tRP_ps = 13125 tRAS_ps = 36000 tWR_ps = 15000 tWTR_ps = 7500 tRFC_ps = 160000 tRRD_ps = 6000 tRC_ps = 49125 refresh_rate_ps = 7800000 tIS_ps = 0 tIH_ps = 0 tDS_ps = 0 tDH_ps = 0 tRTP_ps = 7500 tDQSQ_max_ps = 0 tQHS_ps = 0 FSL DDR>edit c0 opts ECC_mode 0 FSL DDR>edit c0 regs cs0_bnds 0x000000FF FSL DDR>go 2 GiB left unmapped 4 GiB (DDR3, 64-bit, CL=9, ECC off) DDR Chip-Select Interleaving Mode: CS0+CS1 Testing 0x00000000 - 0x7fffffff Testing 0x80000000 - 0xffffffff Remap DDR 2 GiB left unmapped POST memory PASSED Flash: 128 MiB L2: 128 KB enabled Corenet Platform Cache: 1024 KB enabled SERDES: timeout resetting bank 3 SRIO1: disabled SRIO2: disabled MMC: FSL_ESDHC: 0 EEPROM: Invalid ID (ff ff ff ff) PCIe1: disabled PCIe2: Root Complex, x1, regs @ 0xfe201000 01:00.0 - 8086:10d3 - Network controller PCIe2: Bus 00 - 01 PCIe3: disabled In: serial Out: serial Err: serial Net: Initializing Fman Fman1: Uploading microcode version 101.8.0 e1000: 00:1b:21:81:d2:e0 FM1@DTSEC1, FM1@DTSEC2, FM1@DTSEC3, FM1@DTSEC4, FM1@DTSEC5, e1000#0 [PRIME] Warning: e1000#0 MAC addresses don't match: Address in SROM is 00:1b:21:81:d2:e0 Address in environment is 00:e0:0c:00:ea:05 Hit any key to stop autoboot: 0 =>