Project #2: Basic Multiprogramming and System Calls in NACHOS (To be updated)

Due 11:59:59pm on Nov 8, 2011 (extended to Nov 15, 2011)

Contents

Tasks
Test Programs
How to turnin
Issues to Consider
Suggestions
FAQ and Implementation suggestions from a previous TA
NOTE: This assignment is much more difficult than the previous one. It can take over 2 weeks and is very important to start early, for example 3 weeks before the deadline.

The NACHOS code you have been given is capable of executing user application programs, but in an extremely limited way. In particular, at most one user application process can run at a time, and the only system call that has been implemented is the halt system call that shuts down NACHOS. In this assignment, you will correct some of these deficiencies and turn NACHOS into a multiprogramming operating system with a working set of basic system calls.

Tasks

In the first part of this assignment, you are to implement the Fork(), Yield(), Exit(), Exec(), and Join() system calls that act as follows:
The Fork(func) system call creates a new user-level (child) process, whose address space starts out as an exact copy of that of the caller (the parent), but immediately the child abandons the program of the parent and starts executing the function supplied by the single argument. Notice this definition is slightly different from the one in the syscall.h file in Nachos.

The Yield() call is used by a process executing in user mode to temporarily relinquish the CPU to another process.

The Exit(int) call takes a single argument, which is an integer status value as in Unix. The currently executing process is terminated. For now, you can just ignore the status value. Later you will figure out how to get this value to an interested process.

The Exec(filename) system call spawns a new user-level thread (process), but creates a new address space and begins executing a new program given by the object code in the Nachos file whose name is supplied as an argument to the call. It should return to the parent a SpaceId which can be used to uniquely identify the newly created process.

The Join() call waits and returns only after a process with the specified ID (supplied as an argument to that call) has finished.

In the second part of this assignment, you are to implement the file system calls: Creat(), Open(), Read(), Write(), and Close(). The semantics of these calls are specified in syscall.h. You should extend your file system implementations to handle the console as well as normal files.

Test Programs

For this assignment, you will be working on the version of NACHOS in the userprog directory. You will also need to write some simple C test programs, compile them using the MIPS cross compiler, and run them under NACHOS to test that your modifications to NACHOS actually work. In test subdirectory of NACHOS, several sample test programs are provided. The only one that will run properly on an unmodified NACHOS is the halt program. Additional background information is listed here.

In your test programs, you need to exercise the various system calls while we also provide you the sample test programs . Be sure to test each of the system calls, and to try forking up to three processes (since each has a 1024 byte stack, that's all that will fit in NACHOS' 4K byte physical memory) and have them yield back and forth for awhile to make sure everything is working. Since the facility for I/O from user program will also be implemented during this assignment, you may initially have to rely on using debugging printout in the kernel to track what is happening. Use the DEBUG macro for this, and make sure that debugging printout is disabled by default when you submit your code for grading.

Required Output Prints

In order for us to see how your program works, some debugging information must be added in your code. You should print out the following information:

  1. Which system call is called? Whenever a system call is invoked, print:
    System Call: [pid] invoked [call]
    where [pid] is the identifier of the process (SpaceID) and [call] is the name of one of the system calls that you had to implement. Just give the name without parentheses (e.g., Fork, Create, Exit).
  2. How many pages are allocated to the user program when it is loaded? The following line should be printed when a program is loaded:
    Loaded Program: [x] code | [y] data | [z] bss
    where [x], [y] and [z] are the sizes of the code, (initialized) data and bss (uninitialized) data segments in bytes.
  3. Which process has been forked and how many pages are allocated to the forked process? Whenever a new process is forked, print the following line:
    Process [pid] Fork: start at address [addr] with [numPage] pages memory
    where [pid] is the process identifier (SpaceID) of the parent process, [addr] is the virtual address (in hexadecimal format) of the function that the new (child) process starts to execute and [numPage] is the number of pages that the new process gets.
  4. What is the file name in the Exec system call? When a new process should be executed, print the following line:
    Exec Program: [pid] loading [name]
    where [pid] is the identifier of the parent process that executes a new process and [name] is the name of the executable file that is being loaded
  5. When a thread exits, print the following line:
    Process [pid] exits with [status]
    where [pid] is the identifier of the exiting process and [status] is the exit code.
  6. Print error messages when you fail to spawn a process with Fork() or Exec():
    Process [pid] Fork: start at address [addr] with [numPage] pages memory failed
    Exec Program: [pid] loading [name] failed

What to Submit

You have to provide a writeup in a file HW2_WRITEUP in which you have to mention which portions of the assignment you have been able to complete. And for the ones that are incomplete, what is their current status. The description of status for completed exercises or those that you did not start should not be more than 1 line. However, when you submit code that is not working properly (partial solution), make sure that your documentation is very verbose so that the TAs can understand what you have achieved and which parts are missing. This is necessary to receive at least partial credit. When you just submit something that does not work and give no explanations, expect to receive no credit. Also include both group members' names. Make sure that the writeup file is in the top level code directory. The sample test programs for this project are available under ~cs170/nachos-projtest/proj2. You can provide your own test programs.

  1. Go to your 'nachos' directory.
  2. Turn in your 'code' directory with the command:


You can turnin up to 3 times per project and not more than that! The earlier versions will be discarded.

Note: only one turnin per group is accepted!

Issues to Consider

Here is an outline of the the major issues you will have to deal with to make NACHOS into a multiprogrammed system:
In order to handle the various system calls, you will need to modify the ExceptionHandler() function in exception.cc to determine which system call or exception occurred, and to transfer control to an appropriate function. You might want to consider introducing ``stubs'' (functions with empty bodies or with debugging printout so you can tell when they are called) for all the system calls right away, and then postpone their actual implementation until a bit later. This strategy will help you understand better how control is transferred from user mode to system mode when a system call is executed.

The NACHOS code you have been given is extremely simple-minded about memory management. In particular, the constructor function AddrSpace::AddrSpace() simply determines the amount of memory that will be required by the application to be run and then allocates that much space contiguously starting at address zero in physical memory. The page tables (which control the address translation hardware) are set up so that the logical addresses (what the user program sees) are identical to the physical addresses (where the data is actually stored).

The above scheme is inadequate for running more than one application at a time. You will need to design and implement a scheme for allocating and freeing physical memory, and you will need to arrange to set up the page tables so that the logical address space seen by a user application is a contiguous region starting from address zero, even though the data is stored at different physical addresses. You will want to implement a memory management scheme that is flexible enough to extend to virtual memory later in the semester. We suggest implementing a C++ class with methods for allocating and freeing physical memory one page at a time. By setting up the page tables properly, you can give the user application a contiguous logical address space even though each page of actual data might be stored anywhere in physical memory.

The Fork() system call is the most difficult part of this assignment. It is different from the system call Exec in that Fork will start a new process that runs a user function specified by the argument of the call, while Exec will start a process that runs a different executable file. The parameter types for Fork() and Exec() also differ. Fork(func) takes an argument func which is a pointer to a function. The function must be compiled as part of the user program that is currently running. By making this system call Fork(func), the user program expects the following: a new thread will be generated for use by the user program; and this thread will run func in an address space that is an exact copy of the current one. This implementation of Fork makes it possible to have and to access multiple entry points in an executable file.

To make the system call Fork(func) work for the user program, you will need to know how to find the entry point of the function that is passed as the parameter. The parameter convention is determined by the cross-compiler which produces executable code from the user source program. Look at the file exception.cc to see that this entry point, which is an address in the executable code's address space, is already loaded into register 4 when the trap to the exception handler occurs. All you need to do is to insert code into the exception handler (or call a new function of your own) which does the following: set up an address space which is a copy of the address space of the current thread, and load the address that is in register 4 into the program counter. After these steps, use Thread::Fork() to create a new thread, initialize the MIPS registers for the new process, and have both the new and old processes return to user mode. The parent should return to user mode by returning from the exception handler, the child process should continue to run from the address that is now in the program counter, which is the entry point of the function. To implement Fork, you will need to introduce modifications to the AddrSpace class in addrspace.cc so that you can make a ``clone'' of a running user application program. We suggest adding a function that will create a new address space as an exact copy of the original. You will have to allocate additional physical memory for this copy, set up the page tables properly for the new address space, and copy the data from the old address space to the new. Once the physical memory has been allocated and the page tables set up, you will use Thread::Fork() to create a new kernel thread, initialize the MIPS registers for the new process, and then have both the old and the new processes return to user mode. The child process should continue by finishing the Fork() system call. The parent should return to user mode merely by returning from the ExceptionHandler() function.

The Exit() system call should work by calling Thread::Finish(), but only after deallocating any physical memory and other resources that are assigned to the thread that is exiting.

In order to implement the Exec() system call, you will need a method for transferring data (the name of the executable, supplied as the argument to the system call) between the user address space and the kernel. You are not to use functions Machine::ReadMem() and Machine::WriteMem() in machine/translate.cc. Instead, you will have to code your own functions that take into account the address translations described by the page tables to locate the proper physical address for any given logical address. (Recall that strings in C are stored as sequences of characters in successive memory locations, terminated by a null character.)

Once the name of the executable has been copied into the kernel, and the file has been verified to exist, the executable file should be consulted to determine the amount of physical memory required for the new program. This physical memory should be allocated and initialized with data from the executable file, the page tables thread should be adjusted for the new program, the MIPS registers should be reinitialized for starting at the beginning of the new program, and control should return to user mode. File progtest.cc contains a sample for executing a binary program.

If you use ``machine->Run'' to execute a user program, it terminates the current thread. Since Exec() needs to return a space ID to the caller, you should find a way to do that.

NOTE: The object code produced by the MIPS cross-compiler assumes that the data segment begins at the physical address immediately following the text segment. In particular, there is no page alignment, so that if the text segment ends in the middle of a page, then the data segment will start just after it and the page will contain both code and data.

Yield() system call will call Thread::Yield() after making sure to save any necessary state information about the currently executing process.

Be sure to synchronize your code correctly. You will need to put lock operations in your code to ensure that it will work properly. Your locking should be fine grained enough to eliminate any spurious latency problems caused by coarse grained locking. For example, any time a thread accesses the disk or the console it should not hold a lock that would prevent another thread from accessing some other I/O device or piece of data.

You should buffer user file reads in a disk buffer called diskBuffer (defined in system.cc). All of your user-level file I/O must go through the diskBuffer.

You will need to solve a synchronization problem that occurs when multiple processes try to read or write from a file at the same time.

Suggestions

FAQ and Implementation Suggestions

Click here for FAQ and additional implementation suggestions written by a previous TA.