Project #2: Basic Multiprogramming and System Calls in NACHOS

Due May 12, 2003

Contents

Overview

Exercises

Issues to Consider

Suggestions

What to Submit

Grading Criteria

Overview

For this assignment, in the first part, you are to modify NACHOS so that it can run multiple user applications at once. You are to implement the Fork(), Yield(), Exit(), Exec(), and Join() system calls. The detailed specifications for these system calls are given below.

In the second part of this assignment, you are to implement the Creat, Open, Read, Write, and Close system calls. For this assignment, you will be working on the version of NACHOS in the userprog directory. You will also need to write some simple user application programs, compile them using the MIPS cross compiler, and run them under NACHOS to test that your modifications to NACHOS actually work. User application programs are written in ANSI C. To write and compile them go to the test subdirectory of NACHOS. Several sample applications are provided in that directory. The only one that will run properly on an unmodified NACHOS is the halt program. To demonstrate this program, first go to the test directory. Type make halt to cross compile the halt.c source file and create the executable NACHOS user application halt. Then go to the userprog directory and type gmake nachos. When NACHOS is built, type nachos -x halt. This will start NACHOS and ask it to load and run the halt program. You should see a message indicating that NACHOS is halting at the request of the user program.

In brief, what happens when you type nachos -x halt is as follows:

NACHOS starts. The initial thread starts by running function StartProcess() in file progtest.cc.

A new address space is allocated for the user process, and the contents of the executable file halt are loaded into that address space. This is accomplished by the constructor function AddrSpace::AddrSpace() in the file addrspace.cc.

MIPS registers and memory management hardware are initialized for the new user process by the functions AddrSpace::InitRegister() and AddrSpace::RestoreState() in addrspace.cc.

Control is transferred to user mode and the halt program begins running. This is accomplished by the function Machine::Run(), which starts the MIPS emulator.

The system call Halt() is executed from user mode (now running the program halt). This causes a trap back to the NACHOS kernel via function ExceptionHandler() in file exception.cc.

The exception handler determines that a Halt() system call was requested from user mode, and it halts NACHOS by calling the function Interrupt::Halt().

In this assignment, you will also need to know the object file formats for Nachos. This is how NOFF (NACHOS Object File Format) looks like.

-----------
| DATA    |
-----------
| ....    |         -----
-----------             |
| ....    |             |
-----------             |
| bss     | segment     |
-----------             |---- CODE Section
| data    | segment     |
-----------             |
| code    | segment     |
-----------             |
| magic # | 0xbadfad    | 
-----------         -----

--------------
|virtual addr|  points to the location in virtual memory
--------------
|in file addr|  points to a location inside the DATA part of NOFF file
--------------
|size        |  size of a segment in bytes
--------------

When you create user programs and compile them using the MIPS compiler (cross compile) you get COFF (common object file format) file. This is a normal MIPS object (executable) file that has DATA, TEXT and CODE sections. For this file to be runable under Nachos it has to be turned into NOFF. This is done by using coffnoff translator.

Exercises

The Fork(func) system call creates a new user-level (child) process, whose address space starts out as an exact copy of that of the caller (the parent), but immediately the child abandons the program of the parent and starts executing the function supplied by the single argument. Notice this definition is slightly different from the one in the syscall.h file in Nachos.

The Yield() call is used by a process executing in user mode to temporarily relinquish the CPU to another process.

The Exit(int) call takes a single argument, which is an integer status value as in Unix. The currently executing process is terminated. For now, you can just ignore the status value. Later you will figure out how to get this value to an interested process.

The Exec(filename) system call spawns a new user-level thread (process), but creates a new address space and begins executing a new program given by the object code in the Nachos file whose name is supplied as an argument to the call. It should return to the parent a SpaceId which can be used to uniquely identify the newly created process.

The Join() call waits and returns only after a process with the specified ID (supplied as an arguemnt to that call) has finished.

Test your code by creating several user programs that exercise the various system calls while we also provide you the sample test programs . Be sure to test each of the system calls, and to try forking up to three processes (since each has a 1024 byte stack, that's all that will fit in NACHOS' 4K byte physical memory) and have them yield back and forth for awhile to make sure everything is working. Since the facility for I/O from user program will also be implemented during this assignment, you may initially have to rely on using debugging printout in the kernel to track what is happening. Use the DEBUG macro for this, and make sure that debugging printout is disabled by default when you submit your code for grading.

In the second part of this assignment, yoou are to implement the file system calls: Creat, Open, Read, Write, and Close. The semantics of thes calls are specified in syscall.h. You should extend your file system implementations to handle the console as well as normal files.

To support the system calls that access the console device, you will probably find it useful to implement a SynchConsole class that provides the abstraction of synchronous access to the console. The file progtest.cc has the beginning of a SynchConsole implementation.

What to Submit

You should include a file HW2_WRITEUP that explains how you design the code and how your code works, and how to run tests.

You should also indicate what does not work and explain your efforts in order to get partial credits.

Issues to Consider

In order to handle the various system calls, you will need to modify the ExceptionHandler() function in exception.cc to determine which system call or exception occurred, and to transfer control to an appropriate function. You might want to consider introducing ``stubs'' (functions with empty bodies or with debugging printout so you can tell when they are called) for all the system calls right away, and then postpone their actual implementation until a bit later. This strategy will help you understand better how control is transferred from user mode to system mode when a system call is executed.

The NACHOS code you have been given is extremely simple-minded about memory management. In particular, the constructor function AddrSpace::AddrSpace() simply determines the amount of memory that will be required by the application to be run and then allocates that much space contiguously starting at address zero in physical memory. The page tables (which control the address translation hardware) are set up so that the logical addresses (what the user program sees) are identical to the physical addresses (where the data is actually stored).

The above scheme is inadequate for running more than one application at a time. You will need to design and implement a scheme for allocating and freeing physical memory, and you will need to arrange to set up the page tables so that the logical address space seen by a user application is a contiguous region starting from address zero, even though the data is stored at different physical addresses. You will want to implement a memory management scheme that is flexible enough to extend to virtual memory later in the semester. We suggest implementing a C++ class with methods for allocating and freeing physical memory one page at a time. By setting up the page tables properly, you can give the user application a contiguous logical address space even though each page of actual data might be stored anywhere in physical memory.

The Fork() system call is the most difficult part of this assignment. It is different from the system call Exec in that Fork will start a new process that runs a user function specified by the argument of the call, while Exec will start a process that runs a different executable file. The parameter types for Fork() and Exec() also differ. Fork(func) takes an argument func which is a pointer to a function. The function must be compiled as part of the user program that is currently running. By making this system call Fork(func), the user program expects the following: a new thread will be generated for use by the user program; and this thread will run func in an address space that is an exact copy of the current one. This implementation of Fork makes it possible to have and to access multiple entry points in an executable file.

To make the system call Fork(func) work for the user program, you will need to know how to find the entry point of the function that is passed as the parameter. The parameter convention is determined by the cross-compiler which produces executable code from the user source program. Look at the file exception.cc to see that this entry point, which is an address in the executable code's address space, is already loaded into register 4 when the trap to the exception handler occurs. All you need to do is to insert code into the exception handler (or call a new function of your own) which does the following: set up an address space which is a copy of the address space of the current thread, and load the address that is in register 4 into the program counter. After these steps, use Thread::Fork() to create a new thread, initialize the MIPS registers for the new process, and have both the new and old processes return to user mode. The parent should return to user mode by returning from the exception handler, the child process should continue to run from the address that is now in the program counter, which is the entry point of the function. To implement Fork, you will need to introduce modifications to the AddrSpace class in addrspace.cc so that you can make a ``clone'' of a running user application program. We suggest adding a function AddrSpace::Fork(). In brief, calling this function will create a new address space that is an exact copy of the original. You will have to allocate additional physical memory for this copy, set up the page tables properly for the new address space, and copy the data from the old address space to the new. Once the physical memory has been allocated and the page tables set up, you will use Thread::Fork() to create a new kernel thread, initialize the MIPS registers for the new process, and then have both the old and the new processes return to user mode. The child process should continue by finishing the Fork() system call. The parent should return to user mode merely by returning from the ExceptionHandler() function.

The Exit() system call should work by calling Thread::Finish(), but only after deallocating any physical memory and other resources that are assigned to the thread that is exiting.

In order to implement the Exec() system call, you will need a method for transferring data (the name of the executable, supplied as the argument to the system call) between the user address space and the kernel. You are not to use functions Machine::ReadMem() and Machine::WriteMem() in machine/translate.cc. Instead, you will have to code your own functions that take into account the address translations described by the page tables to locate the proper physical address for any given logical address. (Recall that strings in C are stored as sequences of characters in successive memory locations, terminated by a null character.)

Once the name of the executable has been copied into the kernel, and the file has been verified to exist, the executable file should be consulted to determine the amount of physical memory required for the new program. This physical memory should be allocated and initialized with data from the executable file, the page tables thread should be adjusted for the new program, the MIPS registers should be reinitialized for starting at the beginning of the new program, and control should return to user mode. File progtest.cc contains a sample for executing a binary program.

If you use ``machine->Run'' to execute a user program, it terminates the current thread. Since Exec() needs to return a space ID to the caller, you should find a way to do that.

NOTE: The object code produced by the MIPS cross-compiler assumes that the data segment begins at the physical address immediately following the text segment. In particular, there is no page alignment, so that if the text segment ends in the middle of a page, then the data segment will start just after it and the page will contain both code and data.

The Yield() system call will call Thread::Yield() after making sure to save any necessary state information about the currently executing process.

Be sure to synchronize your code correctly. You will need to put lock operations in your code to ensure that it will work properly. Your locking should be fine grained enough to eliminate any spurious latency problems caused by coarse grained locking. For example, any time a thread accesses the disk or the console it should not hold a lock that would prevent another thread from accessing some other I/O device or piece of data.

You should buffer user file reads in a disk buffer called diskBuffer (defined in system.cc). All of your user-level file I/O must go through the diskBuffer.

You will need to solve a synchronization problem that occurs when multiple processes try to read or write from a file at the same time.

Error handling: System calls invoked by a user process in user mode should never ``crash'' NACHOS. This means you should never trust that the arguments supplied for a system call are correct or reasonable. Instead, these arguments should be checked, and if they are incorrect, a failure status should be passed back to the caller. You will want to think of some scheme for accomplishing this.

Build and test incrementally: We suggest not trying to implement everything at once, but rather do a little at a time, testing that each bit you do works before going on to the next modification. This incremental approach seems to work best for operating systems, since it enables you never to be too far from having a version that ``runs''. If you make too many changes all at once, the debugging task to get back to a running system becomes enormous.

Adding source files: Don't be afraid to add new source files to NACHOS, especially if it makes your program more modular. For example, it might be reasonable to add a new source file for the physical memory manager.

Debugging flags and switches: Using the ``-s'' flag to NACHOS along with the ``-x'' flag causes NACHOS to single-step while in user mode. This might be helpful for debugging and understanding. Also, have a look at the file threads/utility.h to see all the code letters that can be supplied along with the ``-d'' flag to enable various kinds of debugging printout from NACHOS. The ``-d m'' option prints out each MIPS instruction as it is executed, which is very helpful for tracing problems with Fork() and Exec().

Incrementing Program Counter after a System Call: Before returning to user mode after a system call, it is necessary to increment the MIPS program counter to point to the next instruction. A system call instruction takes 4 bytes, so you must add 4 to the program counter. If you don't do this, the application making the system call will go into a loop in which it repeatedly makes the same call over and over. This is hard to figure out otherwise, so I'm telling you now so you don't have to.

Required Print-outs

//HW2 Begin of Modification

//HW2 End of Modification
In order for us to see how your program works, some debugging information must be added in your code. You should print out the following information:

1. How many pages are allocated to the user program.
2. Which thread has been forked.
3. How many and which pages are allocated to the forked process.
4. Which system call is called.
5. What is the file name in the Exec system call.
6. When a thread yields/exits, and which thread gets the control.