Class 13
CS 170
23 Nov 2020

On the board
------------

1. Last time
2. TLB
3. Page faults: intro and mechanics
4. Page faults: uses
5. Page faults: costs

---------------------------------------------------------------------------

1. Last time

    --paging, page tables

    --virtual memory on the x86: page table walking

2. TLB
    
    --so it looks like the CPU (specifically its MMU) has to go out
    to memory on every memory reference?

    --called "walking the page tables"

    --to make this fast, we need a cache

    --TLB: translation lookaside buffer

    hardware that stores virtual address --> physical address;
    the reason that all of this page table walking does not slow
    down the process too much
    
    --hardware managed? (x86.)

    --software managed? (MIPS. OS's job is to load the TLB when
    the OS receives a "TLB miss". Not the same thing as a page
    fault.)

    --what happens to the TLB when %cr3 is loaded? [answer: flushed]

    --can we flush individual entries in the TLB otherwise? 
    INVLPG addr

    --how does stuff get in the TLB?

    --answer: hardware populates it

    --questions:

    --does TLB miss imply page fault? (no!)

    --does page fault imply TLB miss? (no!)
        (imagine a page that is mapped read-only. user-level
        process tries to write to it. TLB knows about the mapping,
        so no TLB miss. But this is still a protection violation.
        To cut down on terminology, we will lump this kind of
        violation in with "page fault".)

3. Page faults: intro and mechanics

    Discussed these above. Let's go into a bit more detail...

    Concept:
    
        a reference is illegal, either because it's not mapped in the
        page tables or because there is a protection violation.

        requires the OS to get involved

        this mechanism turns out to be hugely powerful, as we will see.

    Mechanics

        --what happens on the x86?

        --kernel constructs a trap frame and transfers execution to an
        interrupt or trap handler

            ss
            esp    [former value of stack pointer]
            eflags [former value of eflags]
            cs 
          %esp-->   eip    [instruction that caused the trap]
            [error code] 

        %eip is now executing code to handle the trap
            [how did processor know what to load into %eip?]

        error code:
            [ ................................ U/S | W/R | P]
                 unused

            U/S: user mode fault / supervisor mode fault
            R/W: access was read / access was write
            P: not-present page / protection violation

        on a page fault, %cr2 holds the faulting linear address

        --intent: when page fault happens, the kernel sets up the
        process's page entries properly, or kills the process


4. Uses of page faults

    --Best example: overcommitting physical memory (the classical use of
    "virtual memory")

	--your program thinks it has, say, 512 MB of memory, but your
	hardware has only 256 MB of memory

	--the way that this worked is that the disk was (is) used to
	store memory pages

	--advantage: address space looks huge

	--disadvantage: accesses to "paged" memory (as disk pages that
	live on the disk are known) are sllooooowwwww:

	--Rough implementation:

	    --on a page fault, the kernel reads in the faulting page

	    --QUESTION: what is listed in the page structures? how does
	    kernel know whether the address is invalid, in memory,
	    paged, what?

	    --kernel may need to send a page to disk (under what
	    conditions? answer: two conditions must hold for kernel to
	    HAVE to write to disk)

		(1) kernel is out of memory

		(2) the page that it selects to write out is dirty

	--Computers have lots of memory, so less common to hear the
	sound of swapping these days. You would need multiple large memory
	consumers running on the same computer.


    --Many other uses

	--store memory pages across the network! (Distributed Shared
	Memory)

	    --basic idea was that on a page fault, the page fault
	    handler went and retrieved the needed page from some
	    other machine

	--copy-on-write

	    --when creating a copy of another process, don't copy
	    its memory. just copy its page tables, mark the pages as
	    read-only

	    --QUESTION: do you need to mark the parent's pages
	    as read-only as well? 

	    --program semantics aren't violated when programs do
	    reads

	    --when a write happens, a page fault results. at that
	    point, the kernel allocates a new page, copies the 
	    memory over, and restarts the user program to do a write

		--then, only do copies of memory when there is a
		fault as a result of a write

	    --this idea is all over the place; used in fork(), mmap(),
	    etc.

	--accounting

	    --good way to sample what percentage of the memory pages
	    are written to in any time slice: mark a fraction of
	    them not present, see how often you get faults

	--if you are interested in this, check out the paper
	"Virtual Memory Primitives for User Programs", by Andrew W.
	Appel and Kai Li, Proc. ASPLOS, 1991.

        --high-level idea: by giving kernel (or even user-level
        program) the opportunity to do interesting things on page
        faults, you can build interesting functionality


    --Paging in day-to-day use

        --Demand paging: bring program code into memory "lazily"

        --Growing the stack (contiguous in virtual space, probably not
        in physical space)

        --BSS page allocation (BSS segment contains the part of the
        address space with global variables, statically initialized to
        zero. OS can delay allocating and zeroing a page until the
        program accesses a variable on the page.)

        --Shared text 

        --Shared libraries 

        --Shared memory 

5. Page faults: costs

    --What does paging from the disk cost?

	--let's look at average memory access time (AMAT)

	--AMAT = (1-p)*memory access time + p * page fault time,
	where p is the prob. of a page fault.
	
	memory access time ~ 100ns 
	disk access time   ~ 10 ms = 10^7 ns

	--QUESTION: what does p need to be to ensure that paging hurts
	performance by less than 10%?

	1.1*t_M = (1-p)*t_M + p*t_D
	p = .1*t_M / (t_D - t_M) ~ 10^1 ns / 10^7 ns = 10^{-6} 

	so only one access out of 1,000,000 can be a page fault!!

	--basically, page faults are super-expensive (good thing the
	machine can do other things during a page fault)

        Concept is much larger than OSes: need to pay attention to the
	slow case if it's really slow and common enough to matter.


6. Page replacement policies

    --the fundamental problem/question:
	
	--some entity holds a cache of entries and gets a cache miss.
	The entity now needs to decide which entry to throw away. How
	does it decide?

	--make sure you understand why page faults that result from
	"page-not-present in memory" are a particular kind of cache miss

	    --(the answer is that in the world of virtual memory, the
	    pages resident in memory are basically a cache to the
	    backing store on the disk; make sure you see why this claim,
	    about virtual memory vis-a-vis the disk, is true.)

    --the system needs to decide which entry to throw away, which calls
    for a *replacement policy*.
    
    --let's cover some policies....
 
    Specific policies

        * FIFO: throw out oldest (results in every page spending
	the same number of references in memory. not a good idea.  pages
	are not accessed uniformly.)

	* MIN (also known as OPT). throw away the entry that won't
	be used for the longest time. this is optimal.
	
	    our textbook and other references assert its optimality, but
	    they do not prove it. it's a good idea to get in the habit
	    of convincing yourselves of (or disproving) assertions.
	    Here's a proof, under the assumption that the cache is
	    always full:

	    Choose any other scheme. Call it ALT. Now let's sum the
	    number of misses under ALT or OPT, and induct over the
	    number of references. Four cases at any given reference:
	    {OPT hits, ALT hits}, {OPT hits, ALT misses}, {OPT misses,
	    ALT misses}, {OPT misses, ALT hits}. The only interesting
	    case is the last one (in the other cases, OPT does as well
	    or better than ALT, so OPT keeps pace with, or beats, the
	    competition at every reference). Say that the last case
	    happens at a reference, r. By the induction hypothesis, OPT
	    was optimal right up until the *last* miss OPT experienced,
	    at reference, say, r - a.  After that reference, there has
	    been only one miss (the current one, at r). The alternative,
	    ALT, couldn't have done better than OPT up until r-a (by the
	    induction hypothesis). And since r-a, OPT has had only one
	    miss. But ALT could not have had 0 misses between r-a and
	    now because if it did, it means that OPT replaced the wrong
	    entry at r-a (another way to say the same thing: OPT chose
	    which page to evict so that a is maximal). Thus, OPT is no
	    worse than ALT at r. In the remaining cases, OPT is as good
	    or better than ALT in terms of contributing to the number of
	    misses. So by induction, OPT is optimal.

    --evaluating these algorithms (next time)