diff options
Diffstat (limited to 'md/writeup')
| -rw-r--r-- | md/writeup/raspberry5_baremetal_gic_and_timers.md | 354 |
1 files changed, 354 insertions, 0 deletions
diff --git a/md/writeup/raspberry5_baremetal_gic_and_timers.md b/md/writeup/raspberry5_baremetal_gic_and_timers.md new file mode 100644 index 0000000..4f37be4 --- /dev/null +++ b/md/writeup/raspberry5_baremetal_gic_and_timers.md @@ -0,0 +1,354 @@ +title: Raspberry 5 baremetal GIC and timer's +keywords:raspi5,asm,c,kernel,arm64,gic,timer,irq,armstub + +# Raspberry 5 bare metal GIC and timer's + +# Intro + +Bringing up timers for Raspberry PI 5. There is 2 options ARM ones and BCM one. +Then idea is to create irq handlers and handle timer events. Change from previouse +versions of raspberry PI is that BCM timer now using GIC rather then BCM IRQ's. + +## ARM generic timer + +ARM support generic timer's that come out of the box for ARM SoC's + +Linux kerne defines armv8 timers as +```dts +timer { + compatible = "arm,armv8-timer"; + interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) | + IRQ_TYPE_LEVEL_LOW)>, + <GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(4) | + IRQ_TYPE_LEVEL_LOW)>, + <GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(4) | + IRQ_TYPE_LEVEL_LOW)>, + <GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(4) | + IRQ_TYPE_LEVEL_LOW)>, + <GIC_PPI 12 (GIC_CPU_MASK_SIMPLE(4) | + IRQ_TYPE_LEVEL_LOW)>; + }; +``` + +There are one of possibilities to run as timers. + + +## BCM2835 timer + +Timer allows to set value and when the value matches the counter interrupt is rised. +Can be used for system tick's or sheduling events. Its seem's esiers peripheral to +start with. + +### Interrupts + +Linux kernel defines interrupts in device-tree as + +```dts +system_timer: timer@7c003000 { + compatible = "brcm,bcm2835-system-timer"; + reg = <0x7c003000 0x1000>; + interrupts = <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>, + <GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>, + <GIC_SPI 66 IRQ_TYPE_LEVEL_HIGH>, + <GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>; + clock-frequency = <1000000>; +}; +``` + +| GIC_SPI | Actual number | Note | +|---|---|---| +| 64 | 96 | Timer C0 | +| 65 | 97 | Timer C1 | +| 66 | 98 | Timer C2 | +| 67 | 99 | Timer C3 | + +### System Timer Registers + +Address where are mapped was __0x107c003000__ + +| Offset | Register | Name | Description | +|---|---|---|---| +| 0x0 | CS | System Timer Control/Status | | +| 0x4 | CLO | System Timer Counter Lower 32 bits | | +| 0x8 | CHI | System Timer Counter Higher 32 bits | | +| 0xc | C0 | System Timer Compare 0 | | +| 0x10 | C1 | System Timer Compare 1 | | +| 0x14 | C2 | System Timer Compare 2 | | +| 0x18 | C3 | System Timer Compare 3 | | + +### Usage of the driver + +Clear Control registers, is just writing bit's that need to be cleared +```c +#define bcm2835_read32(offset) (*(volatile uint32_t *)(TM_BCM2835_REG_OFFSET+offset)) +#define bcm2835_write32(offset,val) (*(volatile uint32_t *)(TM_BCM2835_REG_OFFSET+offset)) = val + + +void timer_clr_stat() { + bcm2835_write32(BCM2835_CS,0xF); +} +``` + +To set timer event, need to read counter register and add timer interval to that value +```c +void timer_start_c0() { + uint32_t t_lo = bcm2835_read32(BCM2835_CLO); + t_lo += TIMER_INTERVAL; + bcm2835_write32(BCM2835_C0, t_lo); +} +``` + +## GIC + +GIC - Generic Interrupt Controller + +Comparing to other ones that I have worked with this is more complex and support +different modes. All ARM SoC's will have it and in one or other way will need to deal +with it, so its interesting one to check how it works. + +GIC consist from few parts depend on version. RP5 uses GIC-400 + +### GIC Register list + +| Register | Offset | Note | +|---|---|---| +| GICC_CTRL | 0x0 | Enable disable IRQ types, configure IRQ group behaviour | +| GICC_PMR | 0x4 | IRQ priority filter | +| GICC_IAR | 0xC | Read current IRQ to handle | +| GICC_EOIR | 0x10 | Notify that IRQ is handled | +| GICC_APR0 | 0xD0 | State management | +| GICC_APR1 | 0xD4 | State management | +| GICC_APR2 | 0xD8 | State management | +| GICC_APR3 | 0xDC | State management | + +### GICD Register list + +| Register | Offset | Note | +|---|---|---| +|GICD_CTRL | 0x0 | Enable/Disable forward IRQ's | +|GICD_IDR | 0x8 | Identification register | +|GICD_IGROUPR | 0x080 | Set interrupt group | +|GICD_ISENABLER | 0x100 | Set enable IRQ| +|GICD_ICENABLER | 0x180 | Clear enable IRQ | +|GICD_ISPENDR | 0x200 | Set pending interrupt | +|GICD_ICPENDR | 0x280 | Clear pending interrupt | +|GICD_ISACTIVER | 0x300 | Set active interrupt | +|GICD_ICACTIVER | 0x380 | Clear active interrupt | +|GICD_IPRIORITYR | 0x400 | Set IRQ priority | +|GICD_ITARGETSR | 0x800 | Set target CPU | +|GICD_ICFGR | 0xC00 | Interrupt config | + +### Initializing GIC + +### Setting up GIC for use +Setting up GIC. Switch of Controller and Distributor, clear all pending and active statuses. +Set all priority filters, and enable all groups. And then enable Controller and Distributor. + +```c +iowrite32(GICD_CTRL, 0); +iowrite32(GICC_CTRL, 0); + +for (i = 0; i < 16; i++) { + iowrite32(GICD_ICENABLER(i), 0xffffffff); + iowrite32(GICD_ICPENDR(i), 0xffffffff); + iowrite32(GICD_ICACTIVER(i), 0xffffffff); +} +for (i = 0; i < 16; i++) + iowrite32(GICD_IGROUPR(i), 0xffffffff); +for (i = 0; i < 128; i++) { + iowrite32(GICD_IPRIORITYR(i), 0xa0a0a0a0); + iowrite32(GICD_ITARGETSR(i), 0x01010101); +} +for (i = 0; i < 32; i++) + iowrite32(GICD_ICFGR(i), 0x55); +iowrite32(GICD_CTRL, 0x1); +iowrite32(GICC_PMR, 0xff); +iowrite32(GICC_CTRL, 0x3); +``` + +### Draining interrupt queue before use. + +There was some time spent to figure out why IAR report's spuriouse interrupts +so code added that drains all interrupts and restart state. Issue with enabling +all IRQ's there was 3 extra interrupts that for some reason where raised. Probably +something GPU related. And GIC state after few kernel loads in gdb where messed up. + + +drain all pending interrupts +```c +uint32_t val; +while (((val = ioread32(GICC_IAR)) & 0x3ff) < 1020) + iowrite32(GICC_EOIR, val); +``` + +Restart state machine + +```c +iowrite32(GICC_APR0, 0); +iowrite32(GICC_APR1, 0); +iowrite32(GICC_APR2, 0); +iowrite32(GICC_APR3, 0); +``` + +### Enable interrupt handling + +```c +iowrite32(GICD_ISENABLER(irqn/32), 1 << (irqn % 32)); +``` + +### GDB notes +Checking status of GIC_IAR register in gdb + +``` +p/x *0x107fffa00c +``` + +if value is __0x3ff__ its spurious interrupt. I seen this state after loading kernel few times +and couldn't recover from that option was to restart the board or use GICC_APR register's to clear +the state. + +## Execution levels + +Pi 5 launches from EL2 level and that why not possible to configure interrupts. So we +need to modify armstub to be able to do that. + +To check execution level do +```c +static inline uint32_t current_el(void) { + uint64_t el; + __asm__ volatile("mrs %0, CurrentEL" : "=r"(el)); + return (uint32_t)((el >> 2) & 0x3); +} +``` + +On my machine it got reported as EL2 without any stub configuration +there is possible to configure arm stub that will be executed before loading kernel + +``` +armstub=armstubv8.bin +``` + +base example can be found at https://github.com/raspberrypi/tools/blob/master/armstubs/armstub8.S +I tried to modify and run it, Execution level was reported as EL3, seems it failed and need more +investigation then just bare replacment of addresses + +## IRQ handling + +To handle IRQ's need to setup vector table and passed to VBAR register, and irq handling +function + + +### Vector tables + +```arm +.align 11 +sync_exception_handler: + bl kernel_panic + eret + +.macro ventry label + .align 7 + b \label +.endm + +dummy_vector: + b 0x200 + +.align 11 +vector_table: + ventry sync_exception_handler + ventry dummy_vector + ventry dummy_vector + ventry dummy_vector + + //EL1h + ventry dummy_vector + ventry handle_irq + ventry dummy_vector + ventry dummy_vector +``` + +Vector table address can be passed as +for EL2 + +```arm +ldr x1, =vector_table +msr VBAR_EL2, x1 +isb +``` +for EL3 +``` +ldr x1, =vector_table +msr VBAR_EL3, x1 +isb +``` + +It took some time to figure out why EL3 part didn't worked, only after +checking execution level right approach was choosen(it was EL2). Also when I did +attempted to run stub I got EL3 rather then EL2. + +### IRQ handler + +IRQ handler function will be called when generic IRQ is triggered, so it will need +to handle each of the IRQ's is some way + +```c +void irq_handler() { + uint32_t val = ioread32(GICC_IAR); + uint32_t irq = val & 0x3ff; + + if (irq >= 1020) { + return; + } + + pl011_puts("Got IRQ "); + pl011_putu32(irq); + pl011_puts("\r\n"); + bcm2835_write32(BCM2835_CS, 0xf); // clear all four match flags + timer_start_c1(); + + iowrite32(GICC_EOIR, val); +} +``` + +## Sumup + +There is many parts that's need's to be combined to make IRQ's to work + +1. Setting up the vector table +2. Write vector table address +3. Enable the interrupts in GIC +4. Enable global interrupts +5. Set timer +6. Handle interrupt + +Some parts needs to be cleaned up + +## ARMv8 Stub's + +Raspberry Pi provide option to boot custom armstub's + +## Source + +## Links + +https://www.raspberrypi.com/documentation/computers/config_txt.html +https://forums.raspberrypi.com/viewtopic.php?t=371974 +https://docs.amd.com/r/en-US/ug585-zynq-7000-SoC-TRM/Shared-Peripheral-Interrupts-SPI?tocId=jlBXQc~lyPkRtaPCJuxp2Q +https://developer.arm.com/documentation/ihi0048/b/Programmers--Model/CPU-interface-register-descriptions/Interrupt-Acknowledge-Register--GICC-IAR +https://www.scs.stanford.edu/~zyedidia/docs/arm/armv8_baremetal.pdf +https://stackoverflow.com/questions/66962053/why-kernels-arm64-vector-table-is-aligned-with-11 +https://github.com/raspberrypi/tools/blob/master/armstubs/armstub8.S +https://github.com/rsta2/circle/blob/master/boot/config64.txt +https://github.com/rsta2/circle/blob/master/boot/armstub/armstub8.S +https://www.raspberrypi.org/app/uploads/2012/02/BCM2835-ARM-Peripherals.pdf +https://github.com/Utsav-Agarwal/HobOS/blob/main/boot.S +https://developer.arm.com/documentation/ddi0471/b/ +https://developer.arm.com/documentation/ihi0048/b/Programmers--Model/CPU-interface-register-descriptions/CPU-Interface-Control-Register--GICC-CTLR +https://developer.arm.com/documentation/ihi0048/b/Programmers--Model/CPU-interface-register-descriptions/Interrupt-Priority-Mask-Register--GICC-PMR +https://developer.arm.com/documentation/ihi0048/b/Programmers--Model/Distributor-register-descriptions/Distributor-Control-Register--GICD-CTLR +https://developer.arm.com/documentation/ihi0048/b/Programmers--Model/Distributor-register-descriptions/Interrupt-Group-Registers--GICD-IGROUPRn +https://developer.arm.com/documentation/ddi0601/2025-12/External-Registers/GICD-ITARGETSR-n---Interrupt-Processor-Targets-Registers +[/writeup/raspberry5_baremetal_uart.md](/writeup/raspberry5_baremetal_uart.md) + + |