CS170 KOS

Important! -- Before you submit, be sure that your code loads kos_argv at boot. Do not simply hardcode KOS to load 'a.out' when it starts up. Your TAs will be grading your code by running KOS with the -a option. See the bottom of this page for details. (If you are just starting on the lab don't worry if this makes no sense right now - it will soon)

KOS -- An operating system for a simulated MIPS machine

The rest of the labwork in the class is going to focus on writing KOS -- an operating system for a simulated MIPS machine. Obviously we don't have the machinery to let you write a real OS for a real machine, so we're doing the next best thing: we will be writing the operating system for a simulated machine.

One of the problems with this project is that students often get confused concerning which parts of the code are the simulated machine, and which parts are the OS. Students will try to either change or circumnavigate the simulator to fix problems instead of accepting the simulator as a given and working around it in the OS. We will try to keep this from being a problem. To do so, you should try to adopt the following model. The simulator calls the code you write in exactly three situations:

Each time your code can see the user program's register set and main memory as well as any data structures you have defined in your code. So your job is to

And that's it. Nothing more. We'll talk about the best way to organize your code for these three cases as we go along but you should try to keep in mind what the simulator is doing and what the OS (you) is (are) doing.

Basically, things will work as follows. There are two object files for the simulator:

There is also a header file /cs/faculty/rich/public_html/class/cs170/labs/kos_start/start/simulator.h which contains constants and simulator procedure definitions that you will need in your KOS code. The simulator represents the hardware your operating systems is controlling.

The operating system initializes through a C-language subroutine that you write called KOS(). This state of affairs is not so different from an actual operating system. If you were implementing KOS for an actual machine, you'd need to define an initial entry point for the hardware to "jump through" at start-up. In fact, the "boot up" process is when the hardware loads and executes a pre-defined routine that it finds in a place the hardware specifies (e.g. the boot record).

You will link your code to the two object files, and an executable file will be created. When you run the executable, it starts the simulator. Upon instantiation, the simulator calls KOS() and your operating system gains control. From then on, the interaction between the operating system and the simulator is achieved through well defined communication points. The simulator program will call you when the program requires service (which it tells you through an exception) or when a device requires service (through an interrupt). You, however, do not return once called. Instead, you have two ways to return to the simulator after you are finished doing that thing you do. You can call run_user_code() if you want to run (or go back to running) a user program or noop() if you want to "idle" the machine. All exceptions call exceptionHandler() passing in an exception type as an argument, and interruptHandler() which gets an interrupt type as an argument.

Basically, a computer is a collection of resources (memory, CPU, screen, keyboard, etc) that interact in a specific way. It is the job of the operating system to manage these resources in such a way that enables the user make use of the resources with paradigms that are familiar to him/her (like writing programs and executing them). Typically, each resource exports a low-level hardware interface that consists of control registers, data transfer locations, and a set of interrupt response codes. The resources that the simulator provides are as follows:

For the first few labs, you will load user programs into memory using the subroutine load_user_program(char *filename). Note that this does not execute the program -- it simply loads it into memory.

The Console

In all computers, the screen and keyboard are separate devices that interact with the CPU through interrupts. In KOS, there is a single simulated console device that performs these functions through one API. The simulator "hooks" this device up to your standard in and standard out so that you can type things into the console of your simulated MIPS computer, and see the results. It turns out that gdb also works on the simulator since standard in and standard out are the I/O ports. I found this state of affairs to be extremely handy for debugging.

KOS communicates with this console through two procedures and two interrupts:

To try to make for a realistic interface, the console is able to read/write a character around every 100 user operations.

This simulated activity is pretty close to the way things actually work (or used to work) on real machines. consle_write() simulates an interface to the "screen" which can only accept so many characters per second. In a real machine, it would probably let you pass a buffer and a length, but the principle is the same -- you write and then wait for a synchronous interrupt saying that the device is ready to write again. The same is true for console_read() except the interrupt is unsolicited meaning that it does not come in response to something your OS has done. Instead, when the interrupt hits, your OS should know that a character is ready to be read from the keyboard device and should call console_read().

  • There is no timer interrupt in the simulator for this lab.

    User Programs for KOS

    Okay -- this is the weird part. Put on your weird glasses and listen closely. The simulator simulates an old machine that was made by the Digital Equipment Corporation is the late 80's and early 90's. It actually simulates the hardware so well that it can "run" programs compiled for the actual machine. In fact, the rumor is that the developers had the real machine and made sure that any program that ran on it successfully would also run on the simulator.

    The problem is that to run a program on your simulator, it must be a program that would run on this old piece of hardware. Programs compiled for the Intel boxes on campus, however, will not work since they are built to run on Intel running Linux and not MIPS running Ultrix. If we had a working version of the MIPS/Ultrix box, we'd give you all logins and you could compile your own test codes there, copy them to your Intel box, and load them into your simulator to test them.

    We solve this problem using a cross-compiler. The good people at Gnu Software have made a version of gcc that will build programs on our Intel system that would run on the MIPS/Ultrix box if we had one.

    User programs for KOS must be compiled using the cross-compiler. If you use just the regular gcc you will get a version that will run on your system but not the simulated MIPS machine.

    You may use the application programs that your TAs have thoughtfully provided you with test code, found in /cs/faculty/rich/cs170/test_execs/, which are compiled into the executable files in the same directory. There is also a Makefile to show you how this is done.

    Note that the protection of these executables is not r-x. That is because these are not executable on our machines. They are only executable in the context of KOS -- in other words, you can only execute them by loading them into KOS.


    Loading User Programs in KOS

    The basic steps for loading a user program in KOS is:

    When you start initializing the stack, you need to follow a few rules. The stack will grow toward lower memory addresses. Thus, when you want to initialize the stack with certain values (like argv and envp), you should put those values in memory locations at the top end of the user's memory, and then set StackReg to be just below these memory locations. As a convention, the user program will look for argc in main_memory[StackReg+12], argv in main_memory[StackReg+16], and envp in main_memory[StackReg+20]. It will not make use of any memory values greater than main_memory[StackReg] for allocating call frames on the stack.


    Alignment and Pointer Sizes

    As with most modern computers, the MIPS simulator assumes that all 4-byte quantities like integers and pointers are aligned. For this assignment, the MIPS R3000 is configured for a 32-bit address space so all pointers are the same size as integers -- 4 bytes -- but doubles are 8-byte aligned. This will be an issue when you are setting up argc and argv. There is a quirk of the simulator that causes bizarre behavior if you use the top 12 bytes of memory. Thus, do not use them. When you start stuffing stuff into the stack, do not stuff anything into these twelve bytes.

    The Assignment

    Look at start/kos.c. This is a rudimentary KOS implementation that loads the file a.out into user memory and executes it. The a.out program must be very restricted -- the only external procedure that it may call is _exit(). It does not have to call _exit() however, because _exit() is called automatically whenever main() returns.

    Copy /cs/faculty/rich/public_html/class/cs170/labs/kos_start/start/* into your own area, and compile it so that you can run it. Test it on some a.out files that you copy from the /cs/faculty/rich/cs170/test_execs/ directory (try halt and cpu -- not all of the tests will work because they make system calls you haven't implemented yet).

    Now, you are to make the following changes to KOS:

    1. Implement restricted versions of the system calls read() and write(). The syntax of read() and write() are exactly like Linux's read() and write(). Read their man pages if you are not familiar with read() and write() already. Now, implement read() so that it reads from the console whenever the first argument is zero (i.e. standard in), and implement write() so that it writes to the console whenever the first argument is one or two (standard out or standard error). Any other first argument should return with an error.

      The way that the system call driver is written in the simulator is that if you return a negative value to a system call, it will return -1 to the user program, and set the errno value to be the returned value times -1. This is how you can return different errors from system calls. See an old errno.h from Sun OS for an example set of errnos. (As of this writing, Fedora Core's Linux keeps its list of errnos in /usr/include/asm-generic/errno-base.h and /usr/include/asm-generic/errno.h.)

      Remember that if the user specifies an address of x, that is not address x in KOS. It is the user's address x.

      An important fact of OS design is that users can do incorrect things, but the operating system can't. Thus, KOS should be ready for any arguments to read() and write(), and if the arguments are incorrect, it must deal with that gracefully (i.e. return with an error) and not abort or dump core or leave KOS in such a state that future system calls will break.

    2. Change KOS further so that it will call a given file with a given set of arguments. These can be compiled into the program, but in an easy-to-change place (i.e. as global variables in kos.c). Now, not only should KOS load the file and execute it, but it should set up memory so that the program sees the arguments as argc and argv.

    In short, what you hand in should allow cross-compiled codes, like cat, hw, and alot and in /cs/faculty/rich/cs170/test_execs/ to read and write standard in, standard out and standard error and that work with the console. Not all of the programs in in the test directory will work because your version of KOS does not support all of the needed system calls (those made by the stdio library, for example). You should write your own test codes that make calls to read() and write() to test. Your code should also allow KOS to take arguments through the "-a" flag (see below).


    Submission Instructions

    When you are done, you must submit the following mandatory files: You may also choose to submit any or all of the following optional files: When you submit your code you will see the results of a compilation test. This a sanity check for you to make sure that any last second changes you might have made didn't break your submission. It NOT an indication of the correctness of your submission!

    Structuring your code

    A step-by-step description of a way of implementing this lab is in this cookbook (due to Jim Plank). It is not required that you do things this way, but it is in your best interest to follow this set of guidelines -- it shows the design and development philosophy used by an expert programmer, it will relieve you of a lot of design stress, and it will position you well for the future labs.

    Also, you might take advantage of Heloise's Helpful Hints for Lab #1 as she always seems to have advice to lend.


    Appendix -- Various Things


    Simulator Subtleties

    The simulator is built so that when you do something incorrect it will quit. However, some of the error messages will not be very helpful. For example, you are not allowed to return from an interrupt or exception and let the simulator pick up where it left off. An error message will be printed if you try to do so. Instead of returning, you must call run_user_code() or noop(). The former will cause the machine to switch to user mode, load the registers, and begin executing code. The noop() function will put the machine into idle mode and wait for an interrupt to occur. Another example of something else you are not allowed to do is write a character to the console while it is already trying to write one out (that is before the interrupt occurs). For example the following code will not work:

            console_write('H');
    console_write('i');

    It will stop the program with the following message:

            Assertion failed: line 107, file "machine/console.c"
    Abort

    Hopefully the file name will clue you in on where you made an error.


    The end of the user's address space

    The C runtime system expects the three words (each 4-bytes in length) above each stack frame to be unused. Thus you cannot set the stack pointer to the last word of memory because when the C runtime starts, it will try and use these words above the stack pointer and the machine will crash (since you will be addressing not existent memory).

    Thus the first memory location that the stack should use is the end of user memory - 13 bytes. That is, your stack pointer needs to start at a location that is at least 13 bytes smaller (in terms of user address space) than the end of user memory.


    Debug Flags

    There are various flags that you can use when invoking kos. You can list them out by executing kos help. The only one of these that might be of any help is the -d e option. This causes a debug message to be printed out when an exception or interrupt occurs.


    Writing Your Own Programs for KOS

    You may wish to write and compile your own programs to test your KOS while you are developing. These will need to be compiled using the cross-compiler. To go about this you can follow the format presented in

    /cs/faculty/rich/cs170/test_execs/Makefile

    You'll only be able to do simple programs at this point.

    Consider good_test.c:

    main()
    {
      int i;
      char buf[30];
    
      for (i=0;i<5;i++){
        sprintf(buf,"#%d: cs170 is a great class!!\n",i);
        write(1,buf,strlen(buf));
      }
      
    }
    
    

    This is a rather trivial program that writes the sentence to the screen along with the value of i. I compiled it using

    /cs/faculty/rich/public_html/class/cs170/labs/kos_start/Makefile.xcomp

    into good_test. The compilation procedure is quite a bit more complicated than a simple gcc call:

    make -f Makefile.xcomp
    /cs/faculty/rich/cs170/xcomp/bin/decstation-ultrix-gcc -c -I/usr/include
    -I/cs/faculty/rich/cs170/xcomp/include -G0  evil_test.c
    /cs/faculty/rich/cs170/xcomp/bin/decstation-ultrix-ld -G0 -T
    /cs/faculty/rich/cs170/xcomp/lib/noff.ld -N -L/cs/faculty/rich/cs170/xcomp/lib
    -o evil_test.coff /cs/faculty/rich/cs170/xcomp/lib/crt0.o
    /cs/faculty/rich/cs170/xcomp/lib/assist.o evil_test.o
    /cs/faculty/rich/cs170/xcomp/lib/libc.a
    /cs/faculty/rich/cs170/xcomp/lib/libsys.a
    /cs/faculty/rich/cs170/xcomp/bin/coff2noff evil_test.coff evil_test
    numsections 3 
    Loading 3 sections:
    	".text", filepos 0xd0, mempos 0x0, size 0x3090
    	".data", filepos 0x3160, mempos 0x3200, size 0x1200
    	".bss", filepos 0x0, mempos 0x4400, size 0x200
    /bin/rm evil_test.coff
    /cs/faculty/rich/cs170/xcomp/bin/decstation-ultrix-gcc -c -I/usr/include
    -I/cs/faculty/rich/cs170/xcomp/include -G0  good_test.c
    /cs/faculty/rich/cs170/xcomp/bin/decstation-ultrix-ld -G0 -T
    /cs/faculty/rich/cs170/xcomp/lib/noff.ld -N -L/cs/faculty/rich/cs170/xcomp/lib
    -o good_test.coff /cs/faculty/rich/cs170/xcomp/lib/crt0.o
    /cs/faculty/rich/cs170/xcomp/lib/assist.o good_test.o
    /cs/faculty/rich/cs170/xcomp/lib/libc.a
    /cs/faculty/rich/cs170/xcomp/lib/libsys.a
    /cs/faculty/rich/cs170/xcomp/bin/coff2noff good_test.coff good_test
    numsections 3 
    Loading 3 sections:
    	".text", filepos 0xd0, mempos 0x0, size 0x3090
    	".data", filepos 0x3160, mempos 0x3200, size 0x1200
    	".bss", filepos 0x0, mempos 0x4400, size 0x200
    /bin/rm good_test.coff
    

    I included the -f switch so that make would know which makefile I intended to use. After doing many fascinating things which I won't be going over it produces good_test. Now to run it we need to tell KOS what to execute.

    Now we're ready:

    ./kos
    
    KOS booting... done.
    
    
    Probing console... done.
    a.out loaded
    new sp: 1048508
    #0: cs170 is a great class!!
    #1: cs170 is a great class!!
    #2: cs170 is a great class!!
    #3: cs170 is a great class!!
    #4: cs170 is a great class!!
    Program exited with value 5.
    Machine halting!
    
    
    Cleaning up...
    Ticks: total 19572, idle 14500, system 10, user 5062
    Disk I/O: reads 0, writes 0
    Console I/O: reads 0, writes 145
    Paging: faults 0
    
    Success !!

    What sorts of programs can you run on KOS at this point? Very, very simple ones. Basically for loops and simple calls to read and write will just about test the limits of lab 1 KOS. Programs that use system calls that you haven't implemented yet will indeed break. The program evil_test.c will hopefully demonstrate this:

    #include 
    
    main()
    {
      int i;
      char *buf;
    
      buf=(char *)malloc(30*sizeof(char));
    
      for (i=0;i<5;i++){
    
        sprintf(buf,"#%d: CS170 is a great class!!\n",i);
        write(1,buf,strlen(buf));
      }
    
    }
    
    


    Note that this is essentially the same program as we saw in good_test. Instead of a static character buffer, however, I'm using malloc(). This is what I got when I ran it:

    ./kos
    
    KOS booting... done.
    
    
    Probing console... done.
    a.out loaded
    new sp: 1048508
    Machine halting!
    
    
    Cleaning up...
    Ticks: total 50, idle 0, system 10, user 40
    Disk I/O: reads 0, writes 0
    Console I/O: reads 0, writes 0
    Paging: faults 0
    

    I put the -d e flags in there to get KOS to report that it had intercepted an unknown system call, namely that to getpagesize(). It is the call that malloc() makes to getpagesize() which causes my KOS executable problems. You'll deal with this in great detail in the next lab.

    ./kos -d e
    
    KOS booting... done.
    
    
    Probing console... done.
    a.out loaded
    new sp: 1048508
    Unknown system call
    Machine halting!
    
    
    Cleaning up...
    Ticks: total 50, idle 0, system 10, user 40
    Disk I/O: reads 0, writes 0
    Console I/O: reads 0, writes 0
    Paging: faults 0
    
    Notice the "Unknown system call" message.

    How did I figure out it was getpagesize() that caused the halt?

    To figure out what system call is causing a problem use gdb and set a breakpoint at the line in exception.c that prints the debug statement:

    DEBUG('e', "Unknown system call\n");
    
    which is line 80 in my solution, run it, and then print out the variable type.
    gdb kos
    GNU gdb (GDB) Fedora 7.5.1-42.fc18
    Copyright (C) 2012 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later 
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
    and "show warranty" for details.
    This GDB was configured as "x86_64-redhat-linux-gnu".
    For bug reporting instructions, please see:
    ...
    Reading symbols from /cs/faculty/rich/src/cs170.labs/kos-4/kos...done.
    (gdb) b exception.c:80
    Breakpoint 1 at 0x8049a7e: file exception.c, line 80.
    (gdb) run -d e
    Starting program: /cs/faculty/rich/src/cs170.labs/kos-4/kos -d e
    
    KOS booting... done.
    
    
    Probing console... done.
    a.out loaded
    new sp: 1048508
    
    Breakpoint 1, exceptionHandler (which=SyscallException) at exception.c:80
    80				DEBUG('e', "Unknown system call\n");
    Missing separate debuginfos, use: debuginfo-install glibc-2.16-34.fc18.i686
    (gdb) print type
    $1 = 64
    
    This tells you that the system call that was unrecognized in evil_test was 64.

    To figure out which call corresponds to 64 look in the file simulator.h and find the system call numbers.

    #define SYS_getpagesize 64
    
    What happened here is that evil_test calls malloc() which calls the getpagesize() system call on Dec Ultrix in order to know how to allocate dynamic memory for malloc() to use.

    Handling argc and argv

    It will be important to support the passing of arguments to programs running in KOS. Some of the test programs do not take arguments, but most do so you will need to get argument passing to work. There are two features of the development environment that will help with this task.

    The first is a mechanism for passing arguments from the command line when you boot your machine that you wish to be passed to the first program that is executed. Because our operating system does not yet support an execve system call, we have to hardcode one program ("argtest") to be loaded at boot time. The cookbook uses something like this:

    	static char *Argv[5] = { "argtest", "Rex,", "my", "man!", NULL };
    This defines an array of strings of length 5 where each element of the array points to a different string except the 5th, which is NULL. This is the format that various Linux system calls (like execve() which we will learn about in the future) expect when they require arguments to be passed to a program when it is executed. By convention, Linux will use the first element of this array as the name of the program to execute, and will pass the entire array (including the first element) to that program when it is executed. Thus this array indicates that the program "argtest" should be exeuted and it should receive the arguments "Rex" "my" "man!". That is, if you where executing this command from the shell, you'd type
    bash% argtest Rex my man!
    
    and the program "argtest" would find a pointer to the string "Rex" in argv[1], "my" in argv[2], and "man!" in argv[3]. For initial development, you can simply redefine this array each time you want to run a different program and recompile. This gets pretty annoying if we have a lot of user programs we want to test and we have to recompile KOS every time we want to run a new program. To get around this, you have been provided with a global variable 'kos_argv' defined in simulator.h. You can pass a list of arguments when you invoke the simulator with the '-a' option as follows:
    	./kos -a 'argtest Rex, my man!'
    and kos_argv will contain the corresponding argument vector:
    	{ "argtest", "Rex,", "my", "man!", NULL }
    Be sure that your implementation utilizes kos_argv when you turn it in! Your TAs will be testing your code on many different programs and need to be able to pass these in using the '-a' option!

    The second feature allows you to transfer the arguments from either the globally defined Argv[] or from kos_argv[] to the execution stack used by a program. The development enviromnment is so faithful to the hardware that it uses a version of gcc that was available for the Ultrix MIPS machine. The gcc compiler defines a specific convention for initializing both the parameters argc and argv[] that are passed to main() as the first two arguments when a new program is executed.

    The function

    int *MoveArgsToStack(int *registers, char *argv[], int mem_base);
    
    takes a properly formatted argv[] vector (i.e. an array of pointers to characters where each pointer points to a null-terminated array of characters and the last pointer in the array of pointers is NULL) and returns a vector of integers. The function also takes a MIPS registers set (represented as an array of integers) and the base location of where the stack is to be located in memory. The function copies the strings pointed to by argv[] onto the base of the program stack in MIPS memory and sets the stack pointer value in the registers vector accordingly. It returns an array of integers which contains the MIPS user-space addresses for these copied strings.

    The other function that the bootstrap supplies is

    void InitCRuntime(int *user_args, int *registers, char *argv[], int mem_base);
    
    This function must be called after a call to MoveArgsToStack() with the vector of integers that is returned from MoveArgsToStack() as the first argument to InitCRuuntime(). The other arguments must be exactly the same arguments that were passed to MoveArgsToStack().

    For example, you code might look something like

    .
    .
    .
    int registers[NumTotalRegs]; // vector of integers holding MIPS registers set
    int *user_args; // vector of string address passed back from MoveRagsToStack()
    .
    .
    .
    registers[StackReg] = MemorySize - 12; // set SP at the top of memory
    .
    .
    .
    user_args = MoveArgsToStack(registers, kos_argv, 0);
    InitCRuntime(user_args,registers,kos_argv,0);
    .
    .
    .
    
    You can call them back-to-back this way or interpose code between the calls but the arguments must be the same and you can't modify their contents between calls. Note that MoveArgsToStack() will expect that registers[StackReg] will be set with the user-space address of the base of the stack (which is the top of user memory - 12 bytes in this example) before it is called. It uses this value to start placing strings on the user-space stack. Note also that the vector of integers passed back by MoveArgsToStack() is allocated by that routine and freed in InitCRuntime() after it is used. You don't need to deallocate it.

    Note finally that in this lab you will be working on getting only a single program to run using all of user-space memory. For this reason, the memory "base" will be zero. In a future lab you will extend your OS to be able to run multiple processes, each with a different memory base that you will need to pass to these routines. You will still only get a way to launch kos with a single program from the command line but we will explain how to use this feature to run multiple processes subsequently.


    Miscellaneous

    You will want to begin working from the start directory. Don't copy over any of the executables or object files. Rather just keep soft links to them. In case bugs are found, they can be fixed and you will always be referencing the latest versions of the code.

    Gradescope Makefile

    Here is the makefile that we will use on Gradescope to build your solution. Please make sure your file organization matches this makefile.
    
    all: $(EXECUTABLES)
    
    INCL = kos.h
    
    USER    = exception.o kos.o scheduler.o console_buf.o syscall.o
    
    kos: $(USER) $(MAIN) $(LIB) $(INCL)
            $(CC) $(CFLAGS) -o kos $(MAIN) $(USER) $(LIB) 
    
    .c.o: $(INCL)
            $(CC) -c $(CFLAGS) $*.c
    
    clean:
            /bin/rm -f *.o core $(EXECUTABLES)