Showing posts with label mini2440. Show all posts
Showing posts with label mini2440. Show all posts

Saturday, 19 August 2017

Understanding linker scripts and walkthrough of it in MDK-OS.

I wanted to write about a certain complex topic called linker scripts. Although the syntax being simple the topic is very vast. I will be taking an example from my OS. From here it is possible to expand and build on top of it. I will discuss the basics of the topic and will expand as I implement further techniques.

Linker scripts are used by the linker (in GNU toolchain it is called ld). Linker scripts allow us to control the positioning and attributes of object code in the final output file.  The linker "stitches" the various object code files into one single output file with instructions on how to position various "sections" of the program taken from the linker script file. The extension of a linker script is ".ld" or sometimes ".lds". The script is written using the linker command language.

Linker always uses a the linker script. If you don't supply any linker scripts the linker will use an internal linker script which is compiled into the linker executable file. You can check out what the default linker script is by typing:

1
ld --verbose

You can provide a linker script by providing a -T option. In my Makefile for the loader and mdkos I have two variables LOADER_LDSCRIPT and OD_LDSCRIPT with strings -Tloader.lds and -Tmdkos.lds respectively.

As mentioned above the linker script combines different input files into a single output file. These  files are in a special format called object file format. The files are called object files. Each of the object file has among other things called sections. The sections in input files are called input sections. Similarly the section in an output file is called an output section.

Each section in an object file has a name and size.  Most sections also have an associated block of data called section contents.  A section may be marked as loadable, which means that the contents should be loaded into memory when the output file is run. A section with no contents may be allocatable, which means that an area in memory should be set aside, but nothing in particular should be loaded there (in some cases this memory must be zeroed out). A section which is neither loadable nor allocatable typically contains some sort of debugging information.

You can see the different sections of an object file by using objdump with the -h option. For example to view the different sections in the mdk_loader elf file I input:

1
arm-none-eabi-objdump -h mdk_loader.elf

Every object file also has a list of symbols, known as the symbol table. A symbol may be defined or undefined. Each symbol has a name, and each defined symbol has an address, among other information. If you compile a C or C++ program into an object file, you will get a defined symbol for every defined function and global or static variable. Every undefined function or global variable which is referenced in the input file will become an undefined symbol.
You can check the different symbols in the object file using the -t option for the objdump or use nm.
For example:

1
arm-none-eabi-objdump -t mdk_loader.elf

Typically the different sections in your program and the place they reside can be:
  • Constant data: For example this can be const char teststr = "Test string". This type of information can safely be stored in ROM and used in place and need not be copied to say RAM.
  • Initialized variables: For example int testint = 1234. This data my physically reside in RAM, the initial values to be loaded at boot time must be in ROM.
  • Uninitialized variables: For example a declaration such as  int testint;. These need not occupy any space in ROM. The start up code simply needs to allocate sufficient space in RAM for them, the linker needs to know how to resolve references to these variables.
In addition we have
  • Startup code (hardware and C run-time initialization) code. This code is written in assembly and must be located at a specific place in ROM.
  • Application code: This is distinct from startup code and usually doesn't have to reside anywhere specific in the memory map.
Next we come to the difficult and confusing part of linker scripts. The "load memory address" (LMA) and the "virtual memory address"(VMA). Note that VMA has nothing to do with concepts of virtual, physical memory etc. Generally every loadable or allocatable section out section has these two addresses.

Virtual memory address (VMA): This is the address the section will have when the output file is run. In the code this is the address which will be used as a reference by other parts of the code. Hence we would have to move code from the Load Memory Address to the Virtual Memory Address.
Load memory address (LMA): This the address the section will be loaded.
In most cases both the addresses will be the same. The case where they may differ is when say the .data section is loaded into ROM and then copied to RAM when the program starts up. In this case the ROM address would be the LMA and the RAM address is the VMA.

An interesting example I have encountered previously was a device with low RAM probably about 4MB.  This device had various applications to simplify it was less than 4MB but if added together would be greater than 4MB, somewhere around 50 - 100 MB monolithic application. The technique is called overlaying. All the applications were loaded into a huge NOR flash. All the applications had overlapping memory addresses in the RAM i.e. all the applications had references starting from the same address in the RAM. This is called the VMA.

The application is a monolithic code. So the applications had been placed in the incrementing addresses i.e. the section was placed at incrementing address in the NOR flash. Now to load the application from the NOR flash to the RAM there was a small program called the loader. When a user wanted to go to a specific application he would select in the user interface (UI) and the loader would copy the application from the start address i.e. the LMA address and pastes it on to the RAM i.e. the VMA address and the program counter(PC) would jump to that particular address.

We can inform ld where to load various parts of the program in two ways.

The first is to assign names to various memory regions of our device and then direct each code or data section to the appropriate memory region. This is what is followed in my code.

The second method is to start the linker's current memory location counter at a known address (the start address of the first section of the memory to be populated) and emit sections one by one to the current location, manually incrementing this location counter as appropriate in order to skip "holes" in the memory map. The "holes" in the memory map can be peripheral memory mapping etc.

For eg:

1
2
3
4
5
6
7
SECTIONS
{
    . = 0x30000000;
    .text : { *(.text) }
    .data : { *(.data) }
    .bss  : { *(.bss) }
}

In this script we know that the RAM of the S3C2440 starts at 0x30000000. So we set the location counter at that location in the RAM. The line . = 0x30000000 achieves this.

Next we tell ld which sections to include in the output file, where to emit them into memory and which sections of the input files should be mapped. The next 3 lines does this task. These lines basically say "collect all .text sections from the input files and emit them to a section called .text in the output file. Next collect all .data sections from the input file and emit them to a section called .data in the output file. Finally collect all .bss sections from the input file and emit them to a section called .bss in the output file".

Now I will describe some of the linker script examples in the MDK OS.

First we define the memory regions as follows:

1
2
3
4
5
6
7
MEMORY
{
 sram : org = 0x00000000 , len = 0x1000
 /*sdram : org = 0x30000000 , len = 0x4000000*/
 sdram : org = 0x30000000 , len = 0x3F00000 /* 63MB RAM */
 vectors : org = 0x33F00000 , len = 0x100000 /* Last 1MB for the isr handlers */
}


In the above case we have
  1. SRAM at location 0x0000 of size 4KB. 
  2. SDRAM at location 0x30000000 of size 64MB but I have commented it out. Instead I am keeping the SDRAM region size of 63MB reserving the last 1MB.
  3. The last 1MB is reserved for the interrupt vectors and it is the vectors region.
Next my different sections looks as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
.text :
{
 *(.text);
 . = ALIGN(4);
} > sdram


.data :
{
 __data_start__ = .;
 *(.data);
 . = ALIGN(4);
 __data_end__ = .;
} > sdram


In this section we have .text section loaded onto the "sdram" region. After this we also have the .data section loaded onto the "sdram" region.

I have the __data_start__ = . and the __data_end__ =  . which is extern'd in the code. These variables will be filled with the addresses of the start and end of the data section. Please note that the __data_start__ and __data_end__ is loaded with the VMA. In .data section the VMA and LMA is the same.

Next we have the following section:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
.rodata :
{
 __rodata_start__ = .;
 *(.rodata);
 . = ALIGN(4);
 __rodata_end__ = .;
} > sdram

.bss  :
{
 __bss_start__ = .;
 *(.bss); *(COMMON)
 __bss_end__ = .;

 __usr_sys_stack_bottom__ = .;
 . += 0x1000;
 __usr_sys_stack_top__ = .;

 __irq_stack_bottom__ = .;
 . += 0x1000;
 __irq_stack_top__ = .;

 __fiq_stack_bottom__ = .;
 . += 0x1000;
 __fiq_stack_top__ = .;

 __svc_stack_bottom__ = .;
 . += 0x1000;
 __svc_stack_top__ = .;
} > sdram

In the above example I have kept the ".rodata" or the read-only data in the SDRAM. This will be moved to ROM later on.

Next we come to the .bss section which is the data section. All the data is clubbed and kept in the "sdram" memory region. We also setup the user, irq, fiq and svc stack sections each the size of 4KB. We also place markers which will be used in the assembly and C code for setup of stack.

Next we come to the usage of the VMA and LMA concepts and that is in the interrupt handlers. We have the ld script as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
.vector_reloc :
{
 *(.vector_reloc);
} >vectors AT>sdram 

/* Get the lma address for the particular section */
__exception_vector_reloc_startaddr__ = LOADADDR(.vector_reloc);
__exception_vector_reloc_endaddr__ = LOADADDR(.vector_reloc) + SIZEOF(.vector_reloc);

/* 
 * Above SDRAM is where it will be stored in the file but address
 * references will be in the addresses of the isr handler section
 */

.isrhandler :
{
 *(.isrhandler);
} >vectors AT>sdram

__exception_handler_start_addr__ = LOADADDR(.isrhandler);
__exception_handler_end_addr__ = LOADADDR(.isrhandler) +  SIZEOF(.isrhandler);

In this we have the .vector_reloc section at sdram which is the LMA denoted by AT>sdram. The VMA is the vectors memory region which starts from 0x33F00000. Following this section is the .isr_handler section which is similar to above which has the LMA being in the SDRAM and the VMA being in the vectors memory region. We use LOADADDR to get the LMA of the section and SIZE to get the size of the section.

In the previous case the variables __irq_stack_bottom__ etc is loaded with the VMA. Since the LMA and the VMA are the same in it we do not bother to use the LOADADDR and SIZE functions.

How does all this come together?
The address generated and all the code references of the vector_reloc and isr_handler is the VMA. It is stored in the LMA though.The code for exception_vectors is present in os_vector.s in the section vector_reloc. The code snippet is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
.section .vector_reloc,"ax" //Apparent fix for missing section when objcopy is to have allocatable and executable flags-"ax"
//TODO: Understand the reason for the above flag.
.code 32

.globl exception_vectors

exception_vectors:
 ldr pc,=do_handle_reset  //Reset vector
 ldr pc,=do_handle_undef  //Undefined instruction
 ldr pc,=do_handle_swi   //Software Interrupt
 ldr pc,=do_handle_pabt   //Abort prefetch
 ldr pc,=do_handle_dabt  //Abort data
 ldr pc,=do_handle_reserved //Reserved
 ldr pc,=do_handle_irq  //IRQ
 ldr pc,=do_handle_fiq  //FIQ
.end

The code for exception handling is present in the file exception_handler.s and in the section isrhandler. The snippet of the code is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
.section .isrhandler,"ax"

.code 32

.globl do_handle_reset
do_handle_reset:
 b do_handle_reset

.globl do_handle_undef
do_handle_undef:
 b do_handle_undef

.globl do_handle_swi
do_handle_swi:
 b do_handle_swi

.globl do_handle_pabt
do_handle_pabt:
 b do_handle_pabt

.globl do_handle_dabt
do_handle_dabt:
 b do_handle_dabt

.globl do_handle_reserved
do_handle_reserved:
 b do_handle_reserved

.globl do_handle_irq
do_handle_irq:

          ...
          ...

.globl do_handle_fiq
do_handle_fiq:
  b do_handle_fiq
...
...

.end

Please note that do_handle_irq contents and some other unrelated contents are replaced with "..." for clarity.

The objdump of the section is as follows run with the following command:


1
arm-none-eabi-objdump -tDSl bin/mdk_os.elf


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
Disassembly of section .vector_reloc:

33f00000 <exception_vectors>:
exception_vectors():
33f00000: e59ff018  ldr pc, [pc, #24] ; 33f00020 <exception_vectors+0x20>
33f00004: e59ff018  ldr pc, [pc, #24] ; 33f00024 <exception_vectors+0x24>
33f00008: e59ff018  ldr pc, [pc, #24] ; 33f00028 <exception_vectors+0x28>
33f0000c: e59ff018  ldr pc, [pc, #24] ; 33f0002c <exception_vectors+0x2c>
33f00010: e59ff018  ldr pc, [pc, #24] ; 33f00030 <exception_vectors+0x30>
33f00014: e59ff018  ldr pc, [pc, #24] ; 33f00034 <exception_vectors+0x34>
33f00018: e59ff018  ldr pc, [pc, #24] ; 33f00038 <exception_vectors+0x38>
33f0001c: e59ff018  ldr pc, [pc, #24] ; 33f0003c <exception_vectors+0x3c>
33f00020: 33f00040  mvnscc r0, #64 ; 0x40
33f00024: 33f00044  mvnscc r0, #68 ; 0x44
33f00028: 33f00048  mvnscc r0, #72 ; 0x48
33f0002c: 33f0004c  mvnscc r0, #76 ; 0x4c
33f00030: 33f00050  mvnscc r0, #80 ; 0x50
33f00034: 33f00054  mvnscc r0, #84 ; 0x54
33f00038: 33f00058  mvnscc r0, #88 ; 0x58
33f0003c: 33f00098  mvnscc r0, #152 ; 0x98

Disassembly of section .isrhandler:

33f00040 <do_handle_reset>:
do_handle_reset():
33f00040: eafffffe  b 33f00040 <do_handle_reset>

33f00044 <do_handle_undef>:
do_handle_undef():
33f00044: eafffffe  b 33f00044 <do_handle_undef>

33f00048 <do_handle_swi>:
do_handle_swi():
33f00048: eafffffe  b 33f00048 <do_handle_swi>

33f0004c <do_handle_pabt>:
do_handle_pabt():
33f0004c: eafffffe  b 33f0004c <do_handle_pabt>

33f00050 <do_handle_dabt>:
do_handle_dabt():
33f00050: eafffffe  b 33f00050 <do_handle_dabt>

33f00054 <do_handle_reserved>:
do_handle_reserved():
33f00054: eafffffe  b 33f00054 <do_handle_reserved>

33f00058 <do_handle_irq>:
do_handle_irq():
...
...
33f00098 <do_handle_fiq>:
do_handle_fiq():
33f00098: eafffffe  b 33f00098 <do_handle_fiq>

Now that we have all the data we can start analysing the dumps.

Firstly we verify the claim that all code references are using VMA regions. If we see the disassembly of the vector_reloc and isrhandler above we can see that the memory regions (the first column) are using the addresses from the vectors region which starts from 0x33F00000. After this the isrhandler follows which starts from 0x33F00040.

Because all references are in the 0x33F0000 range we have to load the code in that memory range from the part of the RAM pointed to by the LMA to the VMA. To get the address of the LMA we use the following code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
extern char __exception_handler_start_addr__[];
extern char __exception_handler_end_addr__[];

extern char __exception_vector_reloc_startaddr__[];
extern char __exception_vector_reloc_endaddr__[];

static void setup_interrupt_vector_table()
{
/*
 * TODO: Optimize it to remove the extra index variables. Unoptimized only for test purposes.
 *
 */

 char *vector_table = (char *)EXCEPTION_INTERRUPT_VECTOR_TABLE_START;

 /* 
  * Need to get the lma of the code.
  * The __exception_vector_reloc_startaddr__ is the lma i.e. the generated 
  * address in the file. I need to use this as the start address for the 
  * later vectors and handlers.
  */

 char *src = (char *)__exception_vector_reloc_startaddr__; 
      
 uint32_t i = 0;

 for(i = (uint32_t)__exception_vector_reloc_startaddr__; 
   i<(uint32_t)__exception_vector_reloc_endaddr__; 
    i++) {
  *vector_table = *src;
  vector_table++;
  src++;
 }


 /* Continue with the same place for handler source  */
 for(i = (uint32_t)__exception_handler_start_addr__; 
     i<(uint32_t)__exception_handler_end_addr__;
     i++) {
  *vector_table = *src;
  vector_table++;
  src++;
 }

}

We extern the markers __exception_handler_start_addr__, __exception_handler_end_addr__ and __exception_vector_reloc_startaddr__,__exception_vector_reloc_endaddr__ .
The exception handler loading starts after the exception vector loading stops.

We get back to the objdump disassembly to analyse further the addresses. We see the loading to of the PC (Program counter) with the function address of the handler. We take the first example starting at 0x33F0000 which is ldr pc, =do_handle_reset.

The do_handle_reset symbol is located at 0x33F00040. To load this address we see that ldr PC, [PC, #24] (#24 is 0x18) which means load the contents of the memory present at PC+24. We face a small dilemma here. We see that the PC value is 0x33F00000. So the value after addition is 0x33F00018 (#24 is 0x18) but in the code we land to a value 0x59ff018 in that memory location 0x33F00018. Why is this?
According to the ARM guide we have the following:

Reading the program counter

When an instruction reads the PC, the value read depends on which instruction set it comes from:

For an ARM instruction, the value read is the address of the instruction plus 8 bytes. Bits [1:0] of this
value are always zero, because ARM instructions are always word-aligned.


Due to this we have actually have the PC value as 0x33F0000 + 0x8 when we do a read of the PC value in the LDR instruction. Hence PC value will be 0x33F00000 + 0x8 which is 0x33F00008. Next we have the addition of #24 which is 0x18 in hex which equals 0x33F00020. The value of that memory region is placed in the register PC. The value at that location is 0x33F00040. Hence the value of 0x33F00040 is placed in the PC which is address of the function do_handle_reset.

We observe something strange in the disassembly. We see the location 33F00020 has the following in the object dump.

1
33f00020: 33f00040  mvnscc r0, #64 ; 0x40

What does mvnscc mean? Why do we have some instructions present there which does not make no sense? Well it stumped me for sometime and then I realized that it is just a value placed in the memory. The PC loads that value which is the address of the do_handle_reset hence loads the instruction from there. Why it shows an instruction? This is because the disassembler just blindly decodes the value present. How did I come to this conclusion? I simply changed the vectors address to 0x32F0000 which loaded another instruction which had the value 0x32Fxxxxx.

Finally how exactly do I make the interrupt handler jump to the address mentioned in the vectors when ARM states that the interrupt handlers should be in location 0x00000000? I just map the address 0x00000000 to the address 0x33F00000 in the MMU translation table. So when the CPU emits the address 0x00000000 it translates to 0x33F00000.

This concludes the post on linker scripts. I will add any new things to this post if I come across anything interesting or make things even more clearer with examples.

Finally I want to conclude with the memory map of the MDK OS.

Memory Map Documentation:
=========================


+-------------------------+ ----> 0x00000000
|                         |       ^
|   Initial bootloader    |       |---> Stepping stone buffer.
|    (mdk_loader)      |       v
+-------------------------+ ----> 0x00001000
|                         |
|                         |
|  Peripheral memory map  |
|          hole           |
.                         .
.                         .
.                         .
+-------------------------+ ----> 0x30000000
|   mdk_os (.text)        |
|         .               |
|         .               |
|   mdk_os (.data)        |
|         .               |
|         .               |
|   mdk_os (.rodata)      |
|         .               |
|         .               |
|   mdk_os (.bss)         |
|         .               |
|         .               |
|   mdk_os (.stack)       |
.                         .
.                         .
.                         .
.                         .
+-------------------------+
+-------------------------+ ----> 0x33F00000
|                         |
| Interrupt Vector table  |
| (section .vector_reloc) |
|                         |
+-------------------------+ ----> 0x33F00020
|                         |
|                         |
| Interrupt handlers      |
| (section .isrhandler)   |
|                         |
|                         |
+-------------------------+ ----> 0x34000000


Friday, 18 December 2015

Interrupt handling in the MDK OS.

In ARM microprocessors the memory map address 0x00000000 is reserved for the vector table which is a set of 32bit words. When an interrupt occurs the processors suspends normal execution and starts loading instructions from the exception vector table. It is usually contains a form of branch instruction to a particular routine.

The interrupt vector table is as follows:

Vector Address
Reset 0x00000000
Undefined 0x00000004
SWI 0x00000008
PABT 0x0000000C
DABT 0x00000010
Reserved 0x00000014
IRQ 0x00000018
FIQ 0x00000018


In the S3C2440 after a power on reset the initial 4KB of the NAND flash memory will be loaded onto an internal boot SRAM called the "stepping stone" buffer and the boot code present in this memory address will be executed. The loader is flashed onto the NAND flash using supervivi.

Interrupt handling in loader:

The "stepping stone" buffer SRAM memory map address is located at 0x00000000. Hence our MDK loader gets executed from there. The MDK loader has the following code at the start:

.section .text
.code 32
.globl vectors

vectors:
 b reset  /* Reset */
 b fault_state  /* Undefined instruction */
 b fault_state /* Software Interrupt */
 b fault_state  /* Abort prefetch */
 b fault_state  /* Abort data */
 b .  /* Reserved */
 b fault_state /* IRQ */
 b fault_state /* FIQ */

The code is placed in .text section. The addresses in this section is generated from 0x00000000. The fragment of the loader script is below:

MEMORY
{
 sram : org = 0x00000000 , len = 0x1000
 sdram : org = 0x30000000 , len = 0x4000000
}

SECTIONS
{
 .text :
 {
  *(.text);
  . = ALIGN(4);
 } > sram

As shown above the section .text is loaded onto the sram section which has origin from 0x00000000 with the length of 0x1000(4096) or 4KB.

Notice that my reset vector contains a branch to the reset label. The reset code fragment is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
reset:
 /* Start by clearing bss section */
 ldr r1, bss_start
 ldr r2, bss_end
 ldr r3, =0

clear_bss:
 cmp r1,r2
 str r3,[r1],#4
 bne clear_bss

 /* load r13 i.e. stack pointer with stack_pointer */
 ldr r13,stack_pointer

 bl main

Here I load the bss_start and bss_end as present in the linker script file. Next in the clear_bss I compare if r1 i.e. the bs_start has reached r2 i.e the bs_end. I clear the bss by storing r3 in r1 memory content and incrementing it by 4. Then if I have not equaled r2 I continue the loop. Else I load the stack pointer in r13 and branch to main. The main is the main() function in os_main.c file.

Where have I got the stack_pointer,bss_start and bss_end variables from?

The code fragment below explains: 


1
2
3
stack_pointer: .word __stack_top__
bss_start : .word __bss_start__
bss_end : .word __bss_end__

Where did the __stack_top__,__bss_start__ and __bss_end__ come from?

The linker script code explains:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
SECTIONS
{
 .text :
 {
  *(.text);
  . = ALIGN(4);
 } > sram

 .data :
 {
  __data_start__ = .;
  *(.data)
  . = ALIGN(4);
  __data_end__ = .;
 } > sram

 .bss  :
 {
  __bss_start__ = .;
  *(.bss); *(COMMON)
  __bss_end__ = .;

  __stack_bottom__ = .;
  . += 0x300;
  __stack_top__ = .;

 } > sram

Notice that the linker script variables has global visibility. Now we can take the generated address and use it in our code. Notice that the __stack_bottom__ and __stack_top__ has 0x300(768) bytes of space. Please note that we are loading __stack_top__ in r13(SP) as the stack is a descending stack.

We are not handling any other interrupts in the loader. So if there are any interrupts that happens we just jump to a fault state as shown below:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
fault_state:
 ldr r3,GPBCON
 ldr r4,GPBDAT
 ldr r5,GPBUP

 ldr r6,=0x15400
 str r6,[r3]  @Set to output
 ldr r6,=0x00
 str r6,[r4]  @Set the led
 ldr r6,=0x1E0
 str r6,[r5]  @Disable pullup 

 b .

I have setup the LED's to glow so that I understand that I am in a fault state.

This completes interrupt handling in the loader after a Power on Reset. Next we will see how we will handle this in the MDK OS.

Interrupt handling in MDK OS:

In the MDK OS the interrupt handling will be done differently. We face several problems with using the initial vectors to jump to a particular interrupt handling routine. First is that if we want to jump to a routine which is placed in the SDRAM at address 0x30000000 it becomes too far a jump.

So how did I fix this? I enabled the MMU and mapped address 0x00000000 to EXCEPTION_INTERRUPT_VECTOR_TABLE_START which is presently hard coded to 0x33F00000. So now whenever the processor jumps to 0x00000000 it will do an address translation and translates it to 0x33F00000 and executes the content at that address.

So how is the implementation done?

First we visit the code where the exception vectors are written(os_vectors.s).


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
.section .vector_reloc,"ax" //Apparent fix for missing section when objcopy is to have allocatable and executable flags-"ax"

.code 32

.globl exception_vectors

exception_vectors:
 ldr pc,=do_handle_reset  //Reset vector
 ldr pc,=do_handle_undef  //Undefined instruction
 ldr pc,=do_handle_swi   //Software Interrupt
 ldr pc,=do_handle_pabt   //Abort prefetch
 ldr pc,=do_handle_dabt  //Abort data
 ldr pc,=do_handle_reserved //Reserved
 ldr pc,=do_handle_irq  //IRQ
 ldr pc,=do_handle_fiq  //FIQ
.end

The code is put at section .vector_reloc (intuitive name vector relocation).

The exception vector code by itself is very simple. It just loads the PC (Program Counter) register with the different exception handlers.

How is the address generated for the code? It would be EXCEPTION_INTERRUPT_VECTOR_TABLE_START.

How is the above address generation determined? We have to look at the linker script of the MDK OS(mdkos.lds).

First we look the memory section:


1
2
3
4
5
6
7
MEMORY
{
 sram : org = 0x00000000 , len = 0x1000
 /*sdram : org = 0x30000000 , len = 0x4000000*/
 sdram : org = 0x30000000 , len = 0x3F00000 /* 63MB RAM */
 vectors : org = 0x33F00000 , len = 0x100000 /* Last 1MB for the isr handlers */
}

I have defined vectors region starting at 0x33F00000.

Next we see the sections:


  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
SECTIONS
{

 .text :
 {
  *(.text);
  . = ALIGN(4);
 } > sdram


 .data :
 {
  __data_start__ = .;
  *(.data);
  . = ALIGN(4);
  __data_end__ = .;
 } > sdram

/*
 * Note on constant string bug (related to .rodata): 
 * There was a bug initially when printing a string constant would make the 
 * device go into a loop printing nonsense. This was due the fact that .rodata section
 * was left out. Due to this the addresses of the constant was emitted after the interrupt
 * vectors but the actual address of the constant was somewhere in between the file. (it
 * was after the stack setup. All the functions which referred to the string would use
 * the address which was emitted at the end of the isr handlers but the string was sitting
 * way before. It would have worked if after startup the string was moved to the address
 * at the end of the isr handler. Instead of doing this we can create a .rodata section and
 * put in the RAM. Also make sure we don't overwrite the read only section with some 
 * method. We can later write the .rodata to say flash and lock the write and do only a
 * read.
 */
 .rodata :
 {
  __rodata_start__ = .;
  *(.rodata);
  . = ALIGN(4);
  __rodata_end__ = .;
 } > sdram

 .bss  :
 {
  __bss_start__ = .;
  *(.bss); *(COMMON)
  __bss_end__ = .;

  __usr_sys_stack_bottom__ = .;
  . += 0x1000;
  __usr_sys_stack_top__ = .;

  __irq_stack_bottom__ = .;
  . += 0x1000;
  __irq_stack_top__ = .;

  __fiq_stack_bottom__ = .;
  . += 0x1000;
  __fiq_stack_top__ = .;

  __svc_stack_bottom__ = .;
  . += 0x1000;
  __svc_stack_top__ = .;
 } > sdram

 
 .vector_reloc :
 {
  *(.vector_reloc);
 } >vectors AT>sdram 

 /* Get the lma address for the particular section */
 __exception_vector_reloc_startaddr__ = LOADADDR(.vector_reloc);
 __exception_vector_reloc_endaddr__ = LOADADDR(.vector_reloc) + SIZEOF(.vector_reloc);

 /* 
     * Above SDRAM is where it will be stored in the file but address
     * references will be in the addresses of the isr handler section
  */


 .isrhandler :
 {
  *(.isrhandler);
 } >vectors AT>sdram

 __exception_handler_start_addr__ = LOADADDR(.isrhandler);
 __exception_handler_end_addr__ = LOADADDR(.isrhandler) +  SIZEOF(.isrhandler);


 /* 
  * >vma region AT > lma region 
  */

 /* 
  * eg: .data section is linked with LMA in ROM and
  * the VMA pointing to the real RAM versions
  */

 .stab 0 (NOLOAD) : 
 {
  [ .stab ]
 }

 .stabstr 0 (NOLOAD) :
 {
  [ .stabstr ]
 }
}

In line 65 vector_reloc part I tell the linker to generate addresses in the range defined by vectors i.e. from 0x33F00000. This will be the VMA region.

Now how do I know where the code is loaded?
The code is loaded by the loader to address 0x30000000 which is the start of the SDRAM. The code is placed after the .bss section. The __exception_vector_reloc_startaddr__ and __exception_vector_reloc_endaddr__ contains the start and end of the exception handler vector section. So when the code is loaded the place where it be present is 0x30XXXXXX. This will be the LMA region. The code has to be loaded from this region to the EXCEPTION_INTERRUPT_VECTOR_TABLE_START(0x3F000000) region.

The loading of these code is done the following way(setup_interrupt_vector_table(..) in os/mmu.c):


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
static void setup_interrupt_vector_table()
{
/*
 * TODO: Optimize it to remove the extra index variables. Unoptimized only for test purposes.
 *
 */

 char *vector_table = (char *)EXCEPTION_INTERRUPT_VECTOR_TABLE_START;

 /* 
  * Need to get the lma of the code.
  * The __exception_vector_reloc_startaddr__ is the lma i.e. the generated 
  * address in the file. I need to use this as the start address for the 
  * later vectors and handlers.
  */

 char *src = (char *)__exception_vector_reloc_startaddr__; 
      
 uint32_t i = 0;

 for(i = (uint32_t)__exception_vector_reloc_startaddr__; 
   i<(uint32_t)__exception_vector_reloc_endaddr__; 
    i++) {
  *vector_table = *src;
  vector_table++;
  src++;
 }


 /* Continue with the same place for handler source  */
 for(i = (uint32_t)__exception_handler_start_addr__; 
     i<(uint32_t)__exception_handler_end_addr__;
     i++) {
  *vector_table = *src;
  vector_table++;
  src++;
 }

}



In line 17 we get the content from "vectoreloc" start address to end address and we copy it to the vector_table pointer pointing to EXCEPTION_INTERRUPT_VECTOR_TABLE_START i.e. 0x3F000000 address.

Apart from that we continue to copy the contents of the interrupt handlers. The isr handlers are placed right next to the exception handlers.

The interrupt service handlers are placed in file exception_handler.s under the section .isrhandler

The code fragment is as follows:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
.section .isrhandler,"ax"


.code 32


.globl do_handle_reset
do_handle_reset:
 b do_handle_reset

.globl do_handle_undef
do_handle_undef:
 b do_handle_undef

.globl do_handle_swi
do_handle_swi:
 b do_handle_swi

.globl do_handle_pabt
do_handle_pabt:
 b do_handle_pabt

.globl do_handle_dabt
do_handle_dabt:
 b do_handle_dabt

.globl do_handle_reserved
do_handle_reserved:
 b do_handle_reserved


.globl do_handle_irq
do_handle_irq:
 sub lr,lr,#4 @Subtract r14(lr) by 4.
 stmfd sp!, {r0-r12,lr} @Save r0-r12 and lr. 
       @sp! indicates sp will be subtracted by the sizes of the registers saved.
       @Instruction details can be read in ARM System Developers guide book at Pg 65.
 /*
  * Note on disabling and enabling CPU IRQ.
  * ======================================
  * There is no need to disable IRQ when in IRQ mode. When there is 
  * an interrupt the processor switches to IRQ mode with the I bit 
  * enabled which means it is masked.
  *
  * It was tested by printing the cpsr_irq which had the value
  * 0x60000092. The 7th bit is set which means the IRQ flag is set.
  *
  * This is the same case with the FIQ.
  */
 
 ldr r2,INTOFFSET    @Load the INTOFFSET value into r2
 ldr r2,[r2]      @Load the value in the address to r2
 

 ldr r3,=interrupt_handler_jmp_table @Load the address of the interrupt handler jump table.

 mov lr,pc
 ldr pc,[r3,r2,LSL #2] @Load the value which is the interrupt handler jmp table.


// bl handle_irq

 //Clear interrupt source pending

 ldr r2,INTOFFSET    @Load the INTOFFSET value into r2
 ldr r2,[r2]      @Load the value in the address to r2

 mov r3,#1    @move 1 to r3.
 mov r3,r3, LSL r2   @Shift left by INTOFFSET and store it in r3
 
 ldr r4,SRCPND
 str r3,[r4]   @Store the value of r3 in r4 address

 ldr r4,INTPND
 str r3,[r4]   @Store the value of r3 in r4 address

 
 
 ldmfd sp!, {r0-r12,pc}^  @Restore the stack values to r0 and r12. Next restore lr to pc.
       @The ^ indicates the spsr has to copied to cpsr. The cpsr was copied to spsr
       @when the interrupt was generated.
       @The restoration of CPSR will change the mode to whatever mode was
       @present before the interrupt was called.
 

.globl do_handle_fiq
do_handle_fiq:
  b do_handle_fiq


The code in os_vector.s for e.g. where the ldr pc,=do_handle_irq was done has the code of do_handle_irq in the file exception_handler.s which contains the implementation.

This concludes the memory juggling needed to execute the interrupts.

Handling of various IRQ's:

To get an interrupt you have to enable the global IRQ and FIQ in the CPSR register. This is done as follows:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
static void enable_irq_fiq(void)
{
 uint32_t cpsr_val = 0;

 __asm__ __volatile__ (
  "mrs r0,cpsr\n\t"   /* Copy CPSR to r0 */
  "bic r0,r0,#0xC0\n\t"  /* Clear IRQ, FIQ */
  "msr cpsr,r0\n\t"   /* Copy modified value to cpsr */
  "mov %0,r0\n\t"
  : [cpsr_val]"=r"(cpsr_val) /* No output */
  : /* No input */
  : "r0" /* r0 gets clobbered */
 );

 //print_hex_uart(UART0_BA,cpsr_val);
}

An optimization would be to rewrite as a macro.

Next we will go to the actual handling of the interrupt exception. For this we have to turn over to the code in exception_hander.s

In the do_handle_irq we have :



 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
do_handle_irq:
 sub lr,lr,#4 @Subtract r14(lr) by 4.
 stmfd sp!, {r0-r12,lr} @Save r0-r12 and lr. 
       @sp! indicates sp will be subtracted by the sizes of the registers saved.
       @Instruction details can be read in ARM System Developers guide book at Pg 65.
 /*
  * Note on disabling and enabling CPU IRQ.
  * ======================================
  * There is no need to disable IRQ when in IRQ mode. When there is 
  * an interrupt the processor switches to IRQ mode with the I bit 
  * enabled which means it is masked.
  *
  * It was tested by printing the cpsr_irq which had the value
  * 0x60000092. The 7th bit is set which means the IRQ flag is set.
  *
  * This is the same case with the FIQ.
  */
 
 ldr r2,INTOFFSET    @Load the INTOFFSET value into r2
 ldr r2,[r2]      @Load the value in the address to r2
 

 ldr r3,=interrupt_handler_jmp_table @Load the address of the interrupt handler jump table.

 mov lr,pc
 ldr pc,[r3,r2,LSL #2] @Load the value which is the interrupt handler jmp table.


// bl handle_irq

 //Clear interrupt source pending

 ldr r2,INTOFFSET    @Load the INTOFFSET value into r2
 ldr r2,[r2]      @Load the value in the address to r2

 mov r3,#1    @move 1 to r3.
 mov r3,r3, LSL r2   @Shift left by INTOFFSET and store it in r3
 
 ldr r4,SRCPND
 str r3,[r4]   @Store the value of r3 in r4 address

 ldr r4,INTPND
 str r3,[r4]   @Store the value of r3 in r4 address

 
 
 ldmfd sp!, {r0-r12,pc}^         @Restore the stack values to r0 and r12. Next restore lr to pc.
     @The ^ indicates the spsr has to copied to cpsr. The cpsr was copied to spsr
     @when the interrupt was generated.
     @The restoration of CPSR will change the mode to whatever mode was
     @present before the interrupt was called.
 

Before we go in depth into the explanation of the code there is a need to explain the first line of the code.
When an exception occurs the link register is set to a specific address based on the current pc. When an IRQ exception is raised the link register lr points to the last executed instruction plus 8. Care has to be taken to make sure the exception handler does not corrupt the lr because lr is used to return from an exception handler. The IRQ exception is taken only after the current instruction is executed, so the return address has to point to the next instruction i.e. lr-4.

The following has useful addresses for the different exceptions.

Exception Address
Reset
Undefined lr
SWI lr
PABT lr-4
DABT lr-8
Reserved
IRQ lr-4
FIQ lr-4



Next we save the registers from r0 to r12.
Next we get the interrupt offset from the interrupt offset register. After this we load the program counter with the index to the handler in the interrupt_handler_jmp_table.

Later code involves interrupt clean up by setting bits in source pending and interrupt pending registers. After this we restore the values r0 to r12 from the stack and load lr to pc to continue where we left off.

Note on the jump tables:

There are 2 jump tables present. The interrupt_handler_jmp_table and external_interrupt_handler_jmp_table. The 2 tables are array of functions pointers of the type void(*handler)(void).


This completes the generic parts of the interrupt handling by the MDK OS. I will add more details if I see anything lacking.


Restlessness is discontent and discontent is the first necessity of progress. Show me a thoroughly satisfied man and I will show you a failure.  
--Thomas A. Edison
                                               

Wednesday, 3 June 2015

Baremetal OS development

A year ago I had decided that I wanted to try writing a simple OS for my MINI2440 development board. The motivation was that this machine being quite powerful why waste cycles on running a full fledged OS like Linux.

Also I wanted to know how far I can go trying to bring an ARM machine to life with just my baremetal code. Being so close to the metal had always fascinated me. I had written code for micro-controllers such as the MSP430 and the STM32F103ZE and had some experience in bringing up an MCU but never brought up an entire SoC.

So the baremetal OS was born. I named it MDK (microcontroller development Kit) for developers and a kernel the MDK-OS. The idea was to have a development kit for the developers and also to have a small kernel.

This brought me to my first commit here:

https://github.com/mindentropy/s3c2440-mdk/commit/c4a9f316b4c8fd6f7984a1209b5cc473fdd3d13e


Ah the joy of seeing the LED's blink. The initial commit featured a Makefile, a linker script an assembly file for the intial interrupt vector table and the main 'C' file which contained the code to blink the LED.

What a joy it was to watch this simple code run.

In this blog I will be explaining and writing my experiences in writing the OS and the development kit. Hope this will act as a learning kit for people who want to write their own OS.


Writing is nature’s way of showing us how sloppy our thinking is.


How to setup Linux on a MINI2440 development board

Assuming an ubuntu setup and with Pengutronix Linux kernel patches for the mini2440.

RootFS creation using debian multistrap. See simple_config to create a simple 
multistrap configuration. 
 
Use tar with the following options --> -cpjvf --numeric-owner
-c --> Create
-p --> Preserve permissions
-j --> bz2.
-v --> verbose
-f --> use file archive ie. the name of the file.


tar -cpjvf <filename.tar.bz2> . --numeric-owner

Extraction is the same. Make sure you are using sudo as the file permissions usually 
will have root user permissions and the permissions are preserved.

Creation of a Emdebian rootfs.
1
2
3
4
sudo debootstrap --foreign --arch armel <debian_release_codename> <rootfs_directory> http://ftp.debian.org/debian/ Do -> cp /usr/bin/qemu-static-arm <rootfs_directory>/usr/bin
Get a root environment. Do -> sudo -i
To do a second stage install. Do -> LC_ALL=C LANGUAGE=C LANG=C chroot debian_armel_wheezy /debootstrap/debootstrap --second-stage
To trigger post install scripts. Do -> LC_ALL=C LANGUAGE=C LANG=C chroot emdebian_rootfs_wheezy_test dpkg --configure -a
 

 
Reference http://wiki.debian.org/EmDebian/CrossDebootstrap#QEMU.2Fdebootstrap_approach

Copy fstab,udev.conf,udev.rules,ts.conf files from pengutronix to the rootfs sdcard.

Reference inittab
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
id:5:initdefault:

T1::sysinit:/bin/mount -t proc proc /proc
T2::sysinit:/bin/mount -o remount,rw /
T3::sysinit:/bin/mkdir -p /dev/pts
T4::sysinit:/bin/hostname -F /etc/hostname

T5::sysinit:/etc/init.d/rcS
T6::sysinit:/bin/mount -a
#T7::sysinit:/bin/mount -t sysfs sys /sys

T0::respawn:/sbin/getty -L ttySAC0 38400 vt100







Reference passwd

1
2
3
root::0:0:root:/root:/bin/sh
bin:x:1:1:bin:/dev/null:/bin/false
nobody:x:99:99:Unprivileged user:/dev/null:/bin/false



Reference group
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
root:x:0
bin:x:1
nogroup:x:99
floppy:x:1000:
cdrom:x:1001:
tape:x:1002:
lp:x:1003:
tty:x:1004:
video:x:1005:
kmem:x:1006:
audio:x:1007:
disk:x:1008:
dialout:x:1009:
crontab:x:1010:




Reference fstab
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# UNCONFIGURED FSTAB FOR BASE SYSTEM

devpts /dev/pts devpts defaults  0 0
none /tmp   tmpfs defaults,mode=1777,uid=0,gid=0 0 0
none /sys   sysfs defaults   0 0
debugfs /sys/kernel/debug debugfs defaults   0 0
usbfs /proc/bus/usb  usbfs devgid=14,devmode=0660  0 0

none /var/log  tmpfs defaults,mode=0755,uid=0,gid=0 0 0
none /run  tmpfs defaults,mode=0755,uid=0,gid=0 0 0
none /run/lock  tmpfs defaults,mode=0755,uid=0,gid=0 0 0
none /var/tmp  tmpfs defaults,mode=1777,uid=0,gid=0 0 0

/etc/securetty

Add ttySAC0 entry at the end of securetty file.

Copy relevant files like udev rules from pengutronix folders.

Kernel Compilation

Get the kernel from kernel.org. If the patch from pengutronix is 3.9.1(i.e. patch.3.9.1.bz2) download the
3.9.0 kernel and the patch will patch the version to 3.9.1. Also if the patches directory shows 3.9 then 
download the 3.9.0 kernel.

Using quilt to apply the 'series' patch.

Create a sym link called patches pointing to 
../OSELAS.BSP-Pengutronix-Mini2440/configs/platform-friendlyarm-mini2440/patches/linux-3.9/ in the
linux directory.
Create a sym link called series pointing to 
../OSELAS.BSP-Pengutronix-Mini2440/configs/platform-friendlyarm-mini2440/patches/linux-3.9/series in
the linux directory.

To apply the series of patches do -> quilt push -av

Copy kernelconfig to the defconfig
Eg: 

1
cp ../OSELAS.BSP-Pengutronix-Mini2440/configs/platform-friendlyarm-mini2440/kernelconfig-3.9 arch/arm/configs/mini2440_defconfig
Kernel compilation: 

1
2
CROSS_COMPILE=arm-linux-gnueabi- ARCH=arm make mini2440_defconfig
CROSS_COMPILE=arm-linux-gnueabi- ARCH=arm make menuconfig
Kernel config

Device Drivers->Graphics Support->Bootup logo check.
        -> Direct Rendering Manager.
        -> Lowlevel video output switch controls.
         ->Console display driver support -> Framebuffer console support.
         Optional 
          Framebuffer console rotation.
         Mini 4x6 font. (For smaller font).

Device Drivers->Graphics Support->Support for framebuffer devices.
        -> S3C2410 LCD framebuffer support.
        -> S3C2410 lcd debug messages.

Device Drivers->Character Devices->Enable TTY.
        -> Virtual Terminal.
File Systems-> Enable ext2,ext3
Network support -> Networking support -> IP: BOOTP support.
          IP: RARP support.


1
2
3
CROSS_COMPILE=arm-linux-gnueabi- ARCH=arm  INSTALL_MOD_PATH=/mnt/arm make 
CROSS_COMPILE=arm-linux-gnueabi- ARCH=arm  INSTALL_MOD_PATH=/mnt/arm make uImage
CROSS_COMPILE=arm-linux-gnueabi- ARCH=arm  INSTALL_MOD_PATH=/mnt/arm make modules_install
uboot options:
1
2
bootargs=mini2440=5tb rootfstype=ext3 root=/dev/mmcblk0p2 rw bootdelay=rootwait ip=dhcp console=tty0 --> For lcd console output.
bootargs=mini2440=5tb rootfstype=ext3 root=/dev/mmcblk0p2 rw bootdelay=rootwait ip=dhcp console=ttySAC0 --> For serial output.
Bootcmd for sdcard boot:
1
bootcmd=mmcinit ; ext2load mmc 0:1 0x31000000 uImage ; bootm 0x31000000
Bootcmd for tftpboot:
1
bootcmd=tftpboot 0x31000000 192.168.1.101:uImage ; bootm 0x31000000
nfs setup 

1
2
3
mkdir /export
mkdir /export/fs
mkdir /export/kernel
 
/etc/exports 

1
2
3
/export 192.168.1.0/24(rw,fsid=0,insecure,no_subtree_check,async)
/export/fs 192.168.1.0/24(rw,fsid=0,insecure,no_subtree_check,async,no_root_squash)
/export/kernel 192.68.1.0/24(rw,fsid=0,insecure,no_subtree_check,async)
 
Bootcmd for nfs

1
2
3
4
5
setenv bootcmd nfs 0x31000000 192.168.1.x:/nfs/rootfs/boot/uImage \; bootm
setenv bootargs console=ttySAC0,115200 root=/dev/nfs nfsroot\=192.168.1.x:/nfs/rootfs rw ip=dhcp mini2440=5tb
        |

                                                                \----> * Do not use environment variable * 
 
NFS configuration which worked:

1
/nfs/rootfs 192.168.1.0/24(rw,fsid=0,insecure,no_subtree_check,async,no_root_squash)
For ERROR cannot umount add mini2440 to /etc/hosts in your host. For earlyprintk messages. Enable in the kernel. Also put a 
'earlyprintk'(without quotes) in bootargs
 
For eg: 
1
setenv bootargs console=ttySAC0,115200 root=/dev/nfs nfsroot\=192.168.1.x:/nfs/rootfs rw ip=dhcp mini2440=5tb earlyprintk