Project #3: Virtual Memory Management and File-System

Due 11:59:59pm on March 17, 2006

Part I (Virtual Memory)

Overview
Additional Information
Reccomended Implementation Stages

Part II (File System)

Overview
What to Extend
Issues to Consider
Reccomended Implementation Stages

General Info

Testing Strategy
How to Submit
Required Output
Grading Criteria


Overview

For this part of the assignment, you will be working in the vm directory of NACHOS. Your goal is to implement demand-paged virtual memory. A big difference between this assignment and the previous assignments is that you are given absolutely no code to start with, so you have to design and implement it yourself. The main things that you have to do are summarized as follows.

For this assignment, you will need to read and write data to the backing store, which should be a file called SWAP. The swap file should be accessed using the NACHOS OpenFile class. You can use the filesystem stub routines (the default), so that the SWAP file is created as a Unix file.

Implement a page fault handler and a page replacement policy of your choice. You should test your code under various conditions of system load, including one process with an address space larger than physical memory, and several concurrently running processes with combined address spaces larger than physical memory. The sort program in the test directory is an example of a program designed to stress the virtual memory system.

Additional Information

Page tables were used in assignment 2 to simplify memory allocation and to isolate failures from one address space from affecting other programs. In this assignment we will use page tables to tell the hardware which pages are resident in physical memory and which are only resident on the disk. If the valid bit in a particular page table entry is set, the hardware assumes that the corresponding virtual page is loaded into physical memory in the physical page frame specified by the physicalPage field. Whenever the program generates an access to that virtual page, the hardware accesses the physical page. If the valid bit is not set, the hardware generates a page fault exception whenever the program accesses that virtual page. Your exception handler will then find a free physical page frame, read the page in from the backing store (typically a paging file) to that physical page frame, update the page table to reflect the new virtual to physical mapping, and restart the program. The hardware will then resume the execution of the program at the instruction that generated the fault. This time the access should go through, since the page has been loaded into physical memory.

To find a free frame, your exception fault handler may need to eject a cached page from its physical page frame. If the ejected page has been modified in the physical memory, the page fault handling code must write the page out to the backing store before reading the accessed page into its physical page frame. The hardware maintains some information that will help the page fault handler determine which steps need to be taken on a page fault. In addition to the valid bit, every page table entry contains a use and a dirty bit. The hardware sets the use bit every time it accesses the corresponding page; if the access is a write the hardware also sets the dirty bit. Your code may use these bits in its page fault handling code. For example, your code can use the dirty bit to determine if it needs to write an ejected page back to the backing store. When a page is read in from disk, the page fault handler should clear the dirty and use bits in its page table entry. If the page is ever ejected, the page fault handler checks its dirty bit. If the dirty bit is still clear, the copy of the page on disk is identical to the copy in physical memory and there is no need to write the page back to disk before ejecting it.

As with any caching system, performance depends on the policy used to decide which things are kept in memory and which are only stored on disk. On a page fault, the kernel must decide which page to replace; ideally, it will throw out a page that will not be referenced for a long time, keeping pages in memory those that are soon to be referenced. Another consideration is that the operating system may be able to avoid the overhead of writing modified pages to disk inside the page fault handler by writing modified pages to disk in advance. The page fault handler can take advantage of the clean physical page frame to complete subsequent page faults more quickly. FIFO replacement policy is the easiest one to implement, so you may want to start from there to test your page swapping mechanism. But in reality, FIFO replacement policy is rarely used due to its bad performance. You will have to implement some other more efficient replacement policy in your final version, such as FIFO with second chance or LRU.

Recommended Implementation Stages


Adapted from Christopher Kruegel's Notes on cs170-2 Project 3.

For this project, you will primarily be working with files in userprog and machine directories although new files should be placed in teh vm directory. We suggest the following steps and urge you to keep an eye at all times on the data structures you have chosen to store and access information. The hope is that after each stage, your design shall become more concrete and perhaps you shall have a better perspective of what you need to do and how you would like to do it.

Stage 1:
Make sure that you understand the TranslationEntry class as given in machine/translate.h. Ask yourself why each component of a page-table has been included. See the method Machine::Translate in machine/translate.cc to understand how these fields are used.

Stage 2:
Trace how the MIPS simulator (in machine/mipssim.cc) executes instructions. As a part of executing an instruction, the machine accesses main memory for loading the instruction itself (code segment) and possibly for its operands (data/stack segment). Figure out how memory access is implemented and where the virtual to physical translation is performed. It is during the translation process that the machine can determine if the virtual address it is trying to access belongs to a page which does not reside in physical memory. Figure out how, when and where the PageFaultException is thrown.

Stage 3:
In stage 2, you would have figured out how, when and where the PageFaultException is raised. This Exception results in control being passed back to the kernel and is supposed to be handled in a manner similar to system calls. Currently, this exception is not handled. Add code to exception.cc to call a stub routine when this Exception is raised. This will be your page fault handler. As a part of raising the Exception, the processor saves the faulting virtual address in a register that will be used by the kernel to handle the Exception. Note that, at this point, this mechanism is not exercised by program execution as processes are loaded in their entirety and no page faults are generated.

Stage 4:
Figure out how to start a process with none of its pages in memory. For this, you will need to change the code that you wrote in Project 2 for process creation. You may also need to modify the pagetable structure (in the TranslationEntry class) to keep track of pages that are not in memory. Note that you are free to add fields to the this structure (as long as you don't disturb the existing fields). You will need to keep track of the location from which disk-resident pages are to be loaded. Remember that, initially the pages of a process are all in the executable file. Once a page has been brought in to memory, any subsequent flush of this page to disk (during page replacement) should be to backing store (this storage will be created in Stage 6) and not to the executable file. You will also need to allocate space in the backing store for the pages of this process. You can choose to be conservative and allocate space for the entire virtual address space of the process on the backing store at creation time. You can be even more conservative and choose to copy the entire executable file into the allocated space at startup. If you did this, you would need to only concern yourself with moving pages between backing store and the memory during page fault handling.

Stage 5
At this point, you are all set to receive a page fault. We suggest that you make your dummy page fault handler (set up in stage 3) simply print some debugging information and return. Using this scheme, you should make sure that control actually flows to your page fault handler during program execution. You don't service the page fault at this stage. Therefore, you should run into an infinite loop where the machine keeps raising page faults.

Stage 6
Now you are all set to implement a page replacement algorithm.

Page Fault Handling

The page fault handler gets control as a result of the machine raising a page fault exception. The handling of this exception involves the following tasks.
  • Demand-paged virtual memory makes it possible to run a process with only a fraction of its address space resident in physical memory. This allows the address space of a user process to be potentially much larger than physical memory, and it also affords a greater opportunity for concurrent execution by allowing more processes to be running at a time. Demand-paged virtual memory is supported by address translation hardware that checks each access to main memory to see whether the referenced page is currently resident in physical main memory. If the page is resident, the access proceeds in the normal way. If the page is not resident, then a page fault is generated, and the CPU leaves user mode and begins executing a kernel routine whose purpose is to handle such exceptions. This routine, called the page fault handler, must perform the following tasks:
  • If necessary, allocate space on the backing store to receive the contents of the victim page (assuming that it is dirty and needs flushing).
  • Initiate I/O to write the contents of the victim page to the backing store.
  • Adjust the pagetable for the process to which the victim page belongs to reflect the fact that it is no longer resident in memory.
  • Locate the page for which the fault was generated on the backing store; initiate I/O to load the page into the page frame selected in the previous steps.
  • Adjust the pagetable for the faulting process to reflect the fact that the desired page is now resident in memory.
  • Return to user mode and restart the instruction that caused the fault. Make sure to re-execute the instruction for which the page-fault was generated in the first place.
  • Use a single file called SWAP with 512 sectors to implement the backing store. The size of the swap sectors is the same as that of a physical page frame. You should use the stub implementation of the file system already provided with Nachos (look into filesys/filesys.h and filesys/openfile.h). You will need a mechanism to keep track of the used and free sectors in the swap file (similar to the mechanism that keeps track of the allocation of the physical page frames in the previous assignment).

    Recommended Data Structures

    The page fault handler requires some auxiliary data structures to accomplish its task. The following data structures may be useful.
    1. A swap map is needed to keep track of the allocation of space on the backing store. This can be a bitmap with one bit for each sector of backing store.
    2. A table that has one entry for each page of physical memory, giving information about that page, such as the last time it was referenced and and a pointer to the Thread or AddrSpace which has data resident in that page.
    3. Additional field in the TranslationEntry class that makes up an AddrSpace's page table to represent wether or not the page is on disk or in Memory.
    You will also need to consider synchronization issues. You may need a lock (or at least a ``busy'' flag) for each physical memory page, and possibly also for each virtual memory page. These would be to ensure, for example, that two processes don't try to initiate I/O on the same page at the same time. Keep in mind that any time a process sleeps in the kernel, another process can run, so that when the sleeping process wakes up again things might look very different then when it went to sleep. The bottom line is that two system calls may be processed concurrently and the synchronization is needed.

    Other designs are possible; the above data structures are just to give you an idea of how it can be done.

    Once you get this working, you should be able to execute programs normally. Launch multiple processes in Nachos simultaneously (using exec, fork) and test your code under various conditions of system load. Include a test case using one process with an address space larger than physical memory and a test case using several concurrently running processes with combined address spaces larger than physical memory. The sort program in the test directory is an example of a program designed to stress the virtual memory system.


    Optional Stage 7
    Bonus (10% of project grade) Awarded for Sucessfull Completion
    Partial credit recieved if attempted
    In this stage, you will extend your memory structures and page fault handler to support copy-on-write. This stage will require extensive changes to your existing code, so it is recommended that you completly finsh stages 1-6 before attempting the bonus stage.

    In order to prevent a partialy working implementation from dis-abling your existing working implementation achieved in steps 1-6, it is especially important that you wrap _all_ modified code in #define COPYONWRITE statements that will allow the grader to give you credit for stages 1-6 if your copy-on-write scheme is not finished or working by the time the code is turned in.

    A memory manager implementing copy-on-write does not use extra memory space by duplicating a page (as in a fork system call), instead, you:

    1. set the new page table entry to use the same physical page.
    2. mark the physical page as shared by the new page table entry. This probably requires that you maintain a structure to determine whether a physical page is currently shared (together with information that tells you which processes share that page).
    3. set the read only bit in the new page table entry, so that the machine will generate a fault (ReadOnlyException) on attempts to write to this page. If the page wasn't already being shared, also set the read only bit on the page table entry of the other process.
    When handling a fault caused by trying to write to this read-only page, you must:
    1. verify that this is a valid page (as for any other page fault) and that it is shared
    2. allocate a new physical page
    3. copy the contents of the old physical page to the new physical page
    4. update the faulting page table entry to point to the new physical page instead of the old one.
    5. unmark the old physical page as shared by the faulting page table entry
    You should be aware of the following things: You must:
    1. Implement copy-on-write in your virtual memory system.
    2. Update your Fork system call to copy-on-write pages instead of simple copying them.

    Part II (Building a File System)

    The multiprogramming and virtual memory assignments made use of the Nachos file system with the stub version. The last phase of your project is to use the Nachos's own file system and enhance its functionality. Your implementation should be under subdirectory "filesys".

    The first step is to read and understand the partial file system we have written for you under filesys subdirectory. Run the program `nachos -f -cp test/small small' for a simple test case of our code - `-f' formats the emulated physical disk, and `-cp' copies the UNIX file `test/small' onto that disk.

    The files to focus on are:

    Our file system has a UNIX-like interface, so you may also wish to read the UNIX man pages for creat, open, close, read, write, lseek, and unlink (e.g., type "man creat"). Our file system has calls that are similar (but not identical) to these; the file system translates these calls into physical disk operations. One major difference is that our file system is implemented in C++. Create (like UNIX creat), Open (open), and Remove (unlink) are defined on the FileSystem object, since they involve manipulating file names and directories. FileSystem::Open returns a pointer to an OpenFile object, which is used for direct file operations such as Seek (lseek), Read (read), Write (write). An open file is "closed" by deleting the OpenFile object.

    Many of the data structures in our file system are stored both in memory and on disk. To provide some uniformity, all these data structures have a "FetchFrom" procedure that reads the data off disk and into memory, and a "WriteBack" procedure that stores the data back to disk. Note that the in memory and on disk representations do not have to be identical.

    What to Extend

    Your job is to modify the file system to allow the maximum size of a file to be as large as the disk (128Kbytes). In the basic file system, each file is limited to a file size of just under 4Kbytes. Each file has a header (class FileHeader) that is a table of direct pointers to the disk blocks for that file. Since the header is stored in one disk sector with 128 bytes, the maximum size of a file is limited by the number of pointers that will fit in one disk sector. Increasing the limit to 128KBytes will probably but not necessarily require you to implement doubly indirect blocks.

    The second job is to implement extensible files. In the basic file system, the file size is specified when the file is created. One advantage of this is that the FileHeader data structure, once created, never changes. In UNIX and most other file systems, a file is initially created with size 0 and is then expanded every time a write is made off the end of the file. Modify the file system to allow this; as one test case, allow the directory file to expand beyond its current limit of ten files. In doing this part, be careful that concurrent accesses to the file header remain properly synchronized.

    Issues to Consider

    Here are some things you should be sure to get right:

    Notice that for the input program file, you can still use the regular Unix file system to read its content.

    Currently, the basic file system code assumes it is accessed by a single thread at a time. You can still keep this assumption for this part of the assignment while in a real system, synchronization is needed to allow multiple threads to use file system concurrently.

    Recommended Implementation Stages

    Adapted from Christopher Kruegel's Notes on cs170-2 Project 4.

    For this project, you will primarily be working with files in the machine and the filesys directories and using the built in -f debug flag. We suggest the following steps and urge you to keep an eye at all times on the data structures you have chosen to store and access information. The hope is that after each step, your design shall become more concrete and perhaps you shall have a better idea of what you need to do and how you would like to do it.

    Step 1: Understand how the Nachos filesystem operates
    Take a look at the Create and Open functions in filesys.cc. You will later modify your Create and Open system calls to use these functions. See how OpenFile objects are created and opened.

    Step 2: Understand how Nachos implements files
    Take a look at openfile.cc. The OpenFile class uses the FileHeader class. Examine the FileHeader class and see why the file size is limited. Look at the implementation of FileHeader and see what each part is doing.

    Step 4: Modify your file system calls to use Nachos file system
    In project 2, you wrote system calls for file system related operations (Create, Open, Read, Write, and Close). These calls invoked the file system stubs located in filesys.h that were #defined under FILES_STUB. The Nachos that is built under the filesys/ directory uses the calls defined in filesys.h under the #defines for FILESYS.
    Make the necessary modifications to your system calls to allow them to use the FILESYS version of these calls.

    Step 5: Modify Nachos so that you can create larger (up to 128KB) files. This is the size of the virtual DISK (file) formatted and used in Nachos
    This will require you to implement doubly indirect blocks. The use of indirect and doubly indirect blocks to keep track of sectors used for large files is described in class/section Make sure your file header is not larger than one disk sector. It is not acceptable to allocate all the sector pointers in the FileHeader all at once. In other words, each header should only have the capcity to handle the current file size and will need to be allocate more single indirection blocks (and later, double indirection blocks) when the file is extended to a size that requires the exta header block pointers.

    Step 6: Allow the Write system call to extend the size of a file. Currently, if the user tries to write beyond the end of the file,Nachos returns an error. Modify the current implementation so that a write beyond the end of the file extends the size of the file. The exception is that if the file is created to be a given size, your implementation should allocate as many sectors (and block pointers) as required to hold that amount of data. For now this function can allocate all pages that are needed in order to fill the gap between the previous end sector and the new end sector. Gracefully handle the case of the disk being full - the user process should not crash. Instead an error should be returned.

    Step 7: Allow the Directory file to be extended beyond it's static limit of 10 entries. Only the disk size should limit to the number of files it can hold. It is acceptable to just extend the array (by 1/2 or 1/3 it's current size) each time more DirectoryEntries are needed.

    Step 8: Bonus (10% of project grade) Awarded for Sucessfull Completion Allow the files to allocate sectors on a demand basis. Extension of the file should support direct file access with "holes" in the file. When the file is extended, it should not allocate all the blocks in between the old last block and the one being written to. In other words, your index data structure should only point to sectors that have actually had data written into them by some file system operation. It is during this file extension that your fileHeader may need to be extended to point to new blocks of block pointers.
    You can do this without adding any additional data structures to your FileHeader implementation by simply allocating sectors and storing them in the list of sector pointers that corresponds to that offset.

    Examples:

    1. To allocate a sector for byte offset: 14*SectorSize in a file with only 2 sectors currently allocated you would You would store the freshly allocated sector in the 14th element direct sector pointer array.
    2. To allocate a sector for byte offset: (NumDirectPointers+65)*SectorSize in a file with only 2 sectors currently allocated you would You would store the freshly allocated sector in the 3rd element of the first singly indirect IndirectPointerBlock Make sure it is clear why before moving on.
    3. To allocate a sector for byte offset: 125*SectorSize in a file with only 2 sectors currently allocated you would store the freshly allocated sector in the first element of the first singly indirect IndirectPointerBlock In the only Doubly Indirect pointer block. This will only hold true if you have one doubly indirect pointer block that gives you the last 900 sector capacity and use some other means to store the first 124 sectors.
    4. To allocate a sector for byte offset: 1024*SectorSize in a file with only 2 sectors currently allocated you would You would store the freshly allocated sector in the last element of the last singly indirect IndirectPointerBlock In the only Doubly Indirect pointer block.

    Required Output

    Part 1

    Part 2

    Testing Strategy

    This project is a little more complicated than the previous in it's dependencies and build requirements. Please read the following carefully and and follow it closley when attempting to test your code.
    1. Clean out all the .c files in the code/test directory. There might be name conflicts from previous projects and you don't want to run old userprogs.
    2. Copy the test programs in the cs170/Spr03-Section1.test/proj3/* directories to the code/test directory. The filesys/test directory should only be used for Nachos disks, not user progs.
    3. Change the Makefile in your test/ directory to compile the files you copied in.
    4. Test part 1. Test as you would for project 2. You can run the ./nachos in the vm/ or userprog/ directories. Unless other modifications were made, you will be using the unix file system while running in these directories.
    5. Test part 2. In order to test userprogs with the filesys version of nachos, you will need to load the userprog initially run when nachos starts (-x arg on the command line) onto the Nachos filesystem before it can be run. You can do this using the -cp option (see below). This necessary because the filesys/nachos does not have the unix filesystem visible to it. It has no idea where ../test/myprog is.

      usage:

      csil$ pwd
      cs170/nachos/code/filesys
      csil$ ./nachos -f -cp ../test/myFirstProg myFirstProg -x myFirstProg
      

      If there are more than one file referenced in myFirstProg (via Exec), then I will need to pass each on the command line as such:

      csil$ ./nachos -f -cp ../test/myFirstProg myFirstProg -cp 
      ../test/myExecedProg myExecedProg -x myFirstProg
      

      Now the Open and close calls will need to work in a _flat_ namespace there is no "mkdir" in nachos so all files on the Nachos disk are in the same un-named directory. This might mean changing the .c files you are testing part 2 with by removing all references to ../test

    6. If you are implementing the hole filled file support:
      1. Modify Create() syscall implementation to always create files with a size of zero regardless of the argument passed in.
      2. Modify your SWAP file map to allocate sectors of the SWAP file in a random manner. As is, your swap file likely chooses the first free page it finds. As Vika noted, this will not allow holes to be created in the file.
      3. Test your part 1 with your part 2 from within the filesys directory using the usage listed above. Remember that 1/2 of your disk space is now used by a SWAP file that is 512 sectors.

    NOTES:

    You will be able to test part 1 without having part 2 implemented. You will be able to test part 2 without having part 1 implemented. The test programs are meant to allow you to get maximal credit for your working code by isolating the functionality and testing each individually. The part 1 test programs don't require large file support and the part 2 programs don't require working VM.

    You can wrap your part 1 and part 2 implementations in #defines to allow each to be "removed" allowing the other to be tested independently.

    How to Submit

    You have to provide a writeup in a file HW3_WRITEUP in which you have to mention which portions of the assignment you have been able to complete. And for the ones that are incomplete, what is their current status. The description of status for completed exercises or those that you did not start should not be more than 1 line. However, when you submit code that is not working properly (partial solution), make sure that your documentation is very verbose so that the TAs can understand what you have achieved and which parts are missing. This is necessary to receive at least partial credit. When you just submit something that does not work and give no explanations, expect to receive no credit. Also include both group members' names and a short note listing all office hours that both group members can attend. Make sure that the writeup file is in the top level code directory. You may or may not be asked to arrive during that hour for a 10 minute interview (See grading policy for details). The sample test programs for this project are under ~cs170/Spr03-Section1.test/proj3/part1 and ~cs170/Spr03-Section1.test/proj3/part2. You can provide your own test programs.

    1. Go to your 'nachos' directory.
    2. Turn in your 'code' directory with the command:


    You can turnin up to 3 times per project and not more than that! The earlier versions will be discarded.

    Note: only one turnin per group is accepted!

    Grading Criteria

    Project 3 Point Distribution (out of 200 Possible Points)

    120 Points (60%): Implementation 80 Points (40%): Public Test Case Performance

    Bonus Points: