CSEC3616Cybersecurity Engineering

    Memory segments, the stack and function calls

    How the CPU, its registers and main memory cooperate to run a program, how a C program is laid out in memory, and what a stack frame holds when a function is called.

    • Contrast CPU registers with main memory and describe the fetch, decode, execute cycle.
    • Name the memory regions of a C program and say what each one stores.
    • Describe what is saved and restored when one function calls another.
    • State the roles of EBP, ESP and EIP and their 64-bit names.

    15 min readJump to the quiz ↓

    The CPU fetches instructions from memory that also holds data

    Program instructions and data share main memory. Each function call also leaves a return address on the stack. These two facts explain how a buffer overflow can redirect execution.

    Registers and main memory. The slide separates two places that hold data.

    • CPU registers are small and very fast, located inside the CPU. They temporarily hold the data an instruction is working on.
    • Main memory is relatively larger and slower. It sits outside the CPU but is directly addressable, and it holds the data and instructions of a running program.

    The notes add that the CPU does not run instructions straight from a disk. They are loaded into main memory first, then fetched into registers.

    The instruction cycle. The CPU repeats three steps.

    1. Fetch. Retrieve the instruction from memory using the address in the program counter (PC). Increment the PC to point to the next instruction.
    2. Decode. Work out what operation the binary instruction specifies, such as add, move or jump.
    3. Execute. Carry out the decoded instruction on the ALU or another execution unit.

    On x86 the PC is the instruction pointer, EIP. The notes spell the same loop out in five steps (fetch, update the instruction pointer, decode and load data, execute, store back) and note that a jump or a function call changes EIP directly instead of just stepping it forward.

    C separates global data, dynamic allocations and function frames

    Memory segments. The lecture’s point on the segments slide is about security, not layout: the CPU fetches code from memory, where code is stored alongside data. If an attacker can get the CPU to read code from the wrong location, they might be able to control the program.

    The C memory model. The slide gives three main regions.

    • Global/static memory. Variables that live for the whole program, for example const int x = 5;.
    • Heap. Dynamic allocation, where the program controls both allocating and freeing. For example s = (char *) malloc(10);.
    • Stack. Used when function calls are made. By convention it typically grows from high to low memory addresses. It is managed by the compiler, and a stack pointer points to the address of the last item put onto it. It consists of stack frames.

    One stack frame holds the data a function needs in order to execute and return. For void func(){ int x; }, the variable x and anything called from func() make the stack grow and shrink. This lecture concentrates on the stack.

    The notes fill in the picture of a loaded program: a text (code) segment with write permission disabled and a fixed size, data and bss segments for initialised and uninitialised globals and statics, the heap, and the stack.

    A function call saves the context needed to return

    What a function call does. A call jumps to a different address to run a separate block of instructions. Before the new function starts, the current function’s context must be saved so control can come back. The slide names two things to save: the stack pointer and the EIP value, which is the return address.

    At the instruction level the slide walks through the same three steps.

    • Fetch. Read the call instruction from the address EIP points to, then increment the PC (EIP) so it points at the next instruction. That next address is the return address.
    • Decode. The CPU decodes the instruction.
    • Execute. Push the return address (the current EIP) and other information about the current function onto the stack. Then set EIP to the starting address of the called function.

    The notes describe a stack frame as the function’s arguments, its local variables, the saved frame pointer (SFP) and the return address. The SFP restores the old EBP when the function ends, and the return address restores EIP. Their assembly listing shows the standard pattern: call pushes the return address, the function pushes ebp, copies esp into ebp, and subtracts from esp to reserve space for locals. At the end it puts esp back, pops ebp, and ret pops the return address into EIP.

    Base, stack and instruction pointers have distinct roles

    The three registers that manage the stack. CPUs generally have dedicated registers for this. On an Intel architecture:

    RegisterPoints tox86 (32 bit)x86_64 (64 bit)
    Base pointerBase of the stack (high addresses)EBPRBP
    Stack pointerTop of the stack (low addresses)ESPRSP
    Instruction pointerAddress of the next instructionEIPRIP

    Pitfall

    The “base” of the stack is at the high addresses and the “top” is at the low addresses, because the stack grows downward. Writing that the top is the highest address loses the mark. The notes also use the word “grows upward” for the stack and “expands downwards” for the heap, but both are talking about direction on the diagram. Use the lecture slide’s statement: the stack grows from high to low addresses.

    Exam detail

    Know the three-step instruction cycle, the three C regions with one example of each, and what is saved when a function is called (stack pointer and return address). Know the register table cold, including the 64-bit names. Questions in the next topic assume you can say which register the attacker is trying to overwrite.

    Optional aside

    The notes’ practice quiz tests the regions directly. Global and static variables go in the data segment, malloc memory goes in the heap, and the local variables and parameters of a function go on the stack. In the quiz snippet the pointer variable ptr itself is local, so it sits on the stack, while the block that malloc returns sits on the heap.

    Recall

    Where do code and data live, and which cycle, regions and registers govern function calls?
    • Registers are tiny and fast and inside the CPU. Main memory is bigger and slower and holds the running program’s code and data.
    • The cycle is fetch, decode, execute. Code and data share memory, which is what an attacker exploits.
    • C memory: global/static for the program’s lifetime, heap under programmer control, stack for function calls and growing from high to low addresses.
    • A call saves the stack pointer and return address. EBP/RBP is the base, ESP/RSP the top, EIP/RIP the next instruction.

    Check yourself