The first few steps are just so that you can your hands wet and see some of the parts of kos. Step 0 -- We always starting counting with zero, don't we? Before moving on to the next steps, take a moment to review http://www.cs.ucsb.edu/~rich/class/cs170/notes/fdr/index.html http://www.cs.ucsb.edu/~rich/class/cs170/notes/Kthreads/index.html the latter, as far as the API (you will not need to understand the implementation of kthreads). If you don't understand these references because the C looks too unfamiliar, then please review https://sites.cs.ucsb.edu/~rich/class/cs170/notes/C/index.html and brush up on your C before beginning this lab. This recipe assumes that you understand how to use these tools and using kos to learn them will be unpleasant. Also, there are often many ways to accomplish a step in this recipe so it doesn't tell you exactly what to type at each step. Instead, it expects that you understand what each step is doing. Please do not move on to a new step before you understand all of the steps that come before it. If you don't, you will likely come to step that asks you to do something that builds on a previous step assuming you understand it all up to that point and you will need to go back to the beginning and start over (at least with your understanding). Finally, this cook book is an adaptation of several previous cook books written by various expert chefs over the years and it follows, humbly, their excellent work. The methodology is one of incremental progress toward understanding, but not necessarily directly toward a working solution. That is, some steps ask you to put in code so that you can verify your understanding of what is happening that you will later need to remove to make progress. Again, the goal of each step is to understand how parts of your OS need to work rather than to chip away at the code until a solution emerges. Ready? Start by downloading the contents of http://www.cs.ucsb.edu/~rich/class/cs170/labs/kos_start_v2/start to some directory in which you will be working on csil.cs.ucsb.edu. Type make and then ./kos if you see something like KOS booting... done. Probing console... done. booting KOS! Machine halting! you are underway! You should feel free to modify these files to create your solution. Step 1 -- Change the print statement in KOS(). Build it again and run it to verify that you can print from within the OS. Test it out. The system should print your new message and then halt immediately when you run kos. Step 2 -- Remove the SYSHalt(). Instead, insert a noop() there. Test it out -- it should hang. Now, try it again and type into the console. Nothing happens. Is that right? Try it again with the -d e flag. Then look in exception.c and you'll see what's happening each time the console interrupts the CPU. Put a print statement there in exception.c to assure yourself that this is all ok and your expectation is correct. Then remove it. Step 3 -- Now, try writing a character to the console. Do: console_write('H'); and then do a noop() in the main kos() code. What happens (again, use the -d e flag)? Try writing two characters in rapid succession: console_write('H'); console_write('i'); noop(); What happens? Ok, take out the second console_write() and in exception.c, put a console_write('i') call when you catch the ConsoleWriteInt interrupt (before the noop()). Now what happens when you run it? Is that what you expected? Make sure you understand why. Remove all the code you put in for steps 2 and 3. Step 4 -- The way that I am structuring things is as follows. There are going to be many kt threads running in the operating system. For example, there is going to be a thread whose sole job is to read characters from the console and put them into a buffer. Remember how we said that threads allow you to encapsulate functionality? This is it. Likewise, there will be a thread whose sole job is to take characters from a buffer, and write them onto the console. If a kt thread in KOS is able to run, then it should run in preference to executing user code. Only if there is nothing else for the operating system to do should user code be run. The procedure that runs user jobs is the scheduler. It takes a user job off the ready queue and executes it (with run_user_code()). If there is no user job to execute, it should call noop(). What I did in step 4 was set up the scheduler. I created the files called scheduler.c and scheduler.h and added scheduler.o to USER_OBJS in Makefile. In scheduler.h I defined a "PCB" (Process Control Block) struct to contain information about a user process (take a look at this structure). At this point the only thing that this struct contains is the registers for the process. I also define a global dllist called the readyq which will hold processes to run. The scheduler is a procedure called ScheduleProcess() which will check the ready queue. If it's empty, then it calls noop(). Otherwise, it takes the first PCB off the readyq and calls run_user_code() on its registers. Also, there is a global variable called Current_pcb pointing to the PCB of the currently running process (or NULL if noop() is running). I defined the global in scheduler.c and put external references in scheduler.h for both the currently running PCB and the readyq. To start out, I have ScheduleProcess() call noop(). I removed the print statement from ScheduleProcess() and added a call to noop(); Finally, instead of calling noop() in KOS() I call ScheduleProcess() (be sure to include scheduler.h in kos.c). I test it -- this of course does nothing but call noop(), but hey, that's not bad. Step 5 -- Now, here's the subtle part. The only two places where KOS gets control once the first noop() or run_user_code() is called is in either exceptionHandler() or interruptHandler(). What these procedures will do is the following: - save the current state of the user program (if noop() was being run, then nothing needs to be done), - fork off a thread to service the interrupt - call the scheduler, which calls kt_joinall and will run when all the threads are done. To prepare for that, take out all of the noop()'s in exception.c, and instead call ScheduleProcess() at the end of both exceptionHandler() and interruptHandler(). Test this out (it will do nothing, but test out typing into the console and make sure enough interrupts are caught using the -d e flag). Step 6 -- To reiterate, the only place where run_user_code() or noop() will be called is in the scheduler. The only time the scheduler will be called is when all threads are blocked. Now, we're going to try running a user program. The first thing we need to do is to initialize the readyq. In KOS(), as the first executable line, add readyq = new_dllist(); You will need to figure out how to get the appropriate header file included. Next, in scheduler.c there is a routine called InitUserProcess(). What this does is - Clear out memory - Allocate a new PCB. - load the program in filename into memory, and set up the PCB's registers as in the original kos code that came in the start directory. However, instead of calling run_user_code(), you put the PCB at the end of the ready queue and then call kt_exit(). The scheduler will run the code. In your main kos code, call kt_fork(InitUserProcess, (void *)kos_argv). Then put a call to kt_joinall() so that KOS() will get control after InitUserProcess() exits. Then call ScheduleProcess(). You will need to add kt.h to kos.c to get this to compile. Then, inside ScheduleProcess(), change the code to check to see if there is a pcb on the readyq. If the readyq is not empty, then - remove the pcb that is at the head of the readyq (but do not deallocate it) - set Current_pcb to point to this pcb - call run_user_code(Current_pcb->registers) This last call will start what ever process is at the head of the readyq. Note that you will need to figure out how to dequeue things from the readyq to make this work. Try this with the halt program. Copy halt to "a.out" and try kos -a "a.out" Now, copy the cpu-long program into a.out and try it out. It should run and then exit having used some number of user ticks. On my laptop, it was 480054 (see the Ticks: line). Step 7 -- Now, run cpu-long again and type into the console. It will hang. Why? Because if you did things like I did, you didn't save the state of the currently running program upon an interrupt. In my code, the interrupt is processed (by printing out the debugging flag) and then kt_joinall() is called followed by ScheduleProcess(). Go ahead and add kt_joinall() right before ScheduleProcess() in exception.c in both locations. It will still hang because the scheduler sees that nothing is left in the readyq so it runs noop(). Indeed all the interrupts get caught, but the cpu-long program is lost. To fix this, you need to save the state of the program in interruptHandler, and put it back onto the readyq before processing the interrupt. Do this by changing ScheduleProcess() to set Current_pcb = NULL just before it calls noop(). We will use whether Current_pcb is NULL to determine whether to save the user state in the interrupt handler. In the interrupt handler, before the switch() statement check to see if Current_pcb != NULL (indicating that there is a currently running process). If there is a current process, then call examine_registers(Current_pcb->registers) to save the process state and put the Current_pcb back on the readyq. Now, try cpu-long again and type -- it should finish this time! Step 8 -- There is no step 8. Sorry. Step 9 -- Time to implement a system call. The first one we'll implement is write(). In exceptionHandler(), you need a new system call case: SYS_write. When you first get into exceptionHandler(), you should save the registers into your PCB struct using examine_registers() as the very first thing that happens inside the exception handler. Notice that Current_pcb *must* point to the pcb of the currently running process (or you have done something wrong) because you set it to point to the pcb that came off the head of the readyq before you called run_user_code(). Call examine_registers(Current_pcb->registers) to save the state of the process making a system call. Next, you need to set the type of system call which is in registers[4] of the pcb. Add the line type = pcb->registers[4] after your call to examine registers. Do the same for r5 from registers[5]. Next, to handle a system call, you'll fork off a thread. That thread may return instantly, or it may block (e.g. on a read or a write). After forking off the thread, exceptionHandler() will go to the end of its switch statements and call kt_joinall() followed immediately by ScheduleProcess(). This will execute the forked thread, and when all thread activity has blocked, the scheduler will then run after kt_joinall() completes. Put kt_fork(do_write,(void *)Current_pcb) into the switch statement under a case for SYS_write. This will cause the code for do_fork() located in syscall.c to be forked off. The code will not run, though until the exception handle calls kt_joinall(). To return from a system call, you will be in the thread servicing the system call. To return, you will need to: 1 - Set PCReg in the saved registers to NextPCReg. If you don't do this, the process will keep calling the system call. 2 - Put the return value into register 2. 3 - Put the PCB onto the ready queue. 4 - Call kt_exit Add code to do this to SysCallReturn() in scheduler.c. Note that InitUserProcess() has code for adding a pcb to the readyq. When the scheduler runs next, it will resume the process by taking it off the readyq and running it, as before. I implemented a procedure SysCallReturn(PCB, int) which does the above (in schedule.c). The second argument is the return value. To test this, when I added SYS_write, case in exception.c which forks off a new thread which calls do_write() with Current_pcb as the argument (as described above). Note that the Current_pcb is not necessarily the pcb of the process that made the system call (if there are multiple processes) so the do_write() call (and all threads that service system calls) take the pcb as an argument. For now, make do_write() call SysCallReturn(pcb, 0) instead of calling kt_exit(). In other words, when you call write() in a C program, nothing gets written, and 0 is returned to you. You will need to cast arg from do_write() to a pointer to to a struct PCB_struct (called pcb) before the SysCallReturn(). Test all this out by the hw program. You can do this either by copying the hw to a.out and then running ./kos -a "a.out" or by passing the full path to the hw binary itself as ./kos -a "/cs/faculty/rich/cs170/test_execs/hw" When it runs, nothing should happen to the console, and it should return with a value of zero. Change do_write() to call SysCallReturn(PCB, 1) and test it out again. It should return with a value of 1. Step 10 -- Do the same thing with SYS_read (i.e. have it return zero). Test it on the cat program. It should do nothing. Step 11 -- Alter do_write() and do_read() to check their arguments. The MIPS puts arg1 in Current_pcb->registers[5], arg2 in registers[6] and arg3 in registers[7]. For example, if arg1 is not 1 or 2 in do_write(), it should return with -EBADF (see "man errno" on your system). For do_read(), the first argument must be zero in this lab. If arg2 is less than zero, it should return with -EFAULT. You'll be testing it on the "errors" program. Make sure you catch at least all of those errors. Note that what we're doing here is returning from a system call so you'll want to use your SysCallReturn() instead of return Step 12 -- Now, we're going to start working on writing to the console. In do_write(), call console_write() on the first character of arg2. Then call P() on a semaphore for console writing. (I called mine "writeok" -- it should be initialized to zero). Initialize writeok in KOS() just after you initialize readyq. I put the definition of writeok in console_buf.c and an external reference to it in console_buf.h. When the P() unblocks, you'll call SysCallReturn(PCB, 1). Finally, in exception.c, have the ConsoleWriteInt case call V(writeok). What this does is write the first character of the desired buffer, and then return from the system call when it's written. Remember that arg2 of the write call is a user address. You'll need to convert it to a system address before calling console_write. Test it on the hw program. It should print a 'H' and a 't' to the console, and exit with a value of 1. Step 13 -- Now, we'll write the whole buffer, a character at a time. As before, after writing call P(writeok). When that unblocks, increment the number of chars written and if that's it, call SysCallReturn(). Otherwise, call console_write() on the next character, and P(writeok) again. Etc. Notice that pcb->registers[7] contains the number of characters to write. Also, the return value of a call to write is the number of characters written. Make sure you change the value that is returned in SysCallReturn(). Now hw should run as it is designed. It will output: Hello world the write statement just returned 12 and it will exit with a value of 12. Step 14 -- Finally, we need to prepare for the next labs. In subsequent labs there will be more than one program in memory. That means that more than one program may want to write to the console. Of course, you only want one of these to work at a time. Therefore, you need a second semaphore which I call (writers). This is initialized to one and I put the definitions and external refs in console_buf.c and console_buf.h respectively. In do_write() instead of calling console_write() for the first time, call P(writers). When the P() call returns, you then go into the console_write loop. When you return from the system call, call V(writers). Have a similar setup for reading named readers. This ensures that only one process will write to the console at a time. Test it again on hw.c. Although you won't see any difference from before, you'll be well prepared for the next lab. Step 15 -- Now test the errors program. Your output should be: First error: Bad file number Second error: Bad address Third error: Invalid argument Fourth error: Bad file number Fifth error: Bad address Sixth error: Bad file number If six errors appear, test successful. Step 16 -- Our next step is to actually do something with the characters from the console. What we're going to do is have a console read buffer. I'm making mine 256 integers (you'll see why integers later). It has three semaphores associated with it: nelem, nslots, and consoleWait. What it does is repeat the following: - Call P() on consoleWait. It will unblock when a character is ready to read from the console. - Call P() on nslots. This semaphore should be initialized to the number of slots in the circular buffer. If the buffer is full, it will block until the do_read() call drains it a bit. - When it unblocks, it reads the character from the console and puts it in the buffer. Because the P() on nslots is passed it knows that there is space in the buffer. The do_read() call will need to call V() on nslots every time it remove a character from the buffer so that the semaphore counts the number of available slots correctly. Write this code using the kt threads library and then have the main kos() program fork off one of these threads before calling kt_joinall(). I put the code for this read thread in console_buf.c Moreover, when processing the ConsoleReadInt interrupt, call V() on consoleWait. To test it, print out the contents of the buffer before calling SYSHalt(). Test it by running the cpu-long program and typing into the console. When cpu-long exits, the buffer should contain what you typed into the buffer. Next, try putting over 256 characters into the console (do this by pasting from your mouse), and see if the code still works, and if the buffer just contains the first 256 characters. You'll want to use a circular queue for the console buffer (i.e. head and tail pointers). Step 17 -- The read system call is a little different from the write system call. What you're going to do is read characters from the above buffer, blocking if necessary. First, call P(nelem). When that unblocks, you'll copy a character from the console buffer to the user's buffer and return if the number of characters matches the size. Otherwise, you call P(nelem) again. Also, every time you take a character out of the circular buffer you need to call V() on nslots so that the console read thread will "know" that there is another free slot. Step 18 -- My console read buffer is a circular queue composed of ints so that EOF can be detected. If the console character is -1, then that represents the end of file. I deal with this in do_read(). If it gets a -1 from the console buffer, it returns from the read instantly, throwing away the -1. In that way procedures may keep reading until it returns something less than what they ask for, at which point they know they have hit the EOF. Insert this functionality. Now you can run the "cat" program and terminate it by typing ^D. Step 19 -- Ok! At this point, everything should be working. All that is left to do is some testing. Well -- maybe a lot of testing. Try the argtest program with ./kos -a "/cs/faculty/rich/cs170/test_execs/argtest Rex, my man!" You should see something like &argc is -->1048440<-- argc is -->4<-- argv is -->1048488<-- envp is -->0<-- argv[0] is (1048522) -->/cs/faculty/rich/cs170/test_execs/argtest<-- argv[1] is (1048517) -->Rex,<-- argv[2] is (1048514) -->my<-- argv[3] is (1048509) -->man!<-- Machine halting! Next, try running cat with an input file. Create a text file called cat-input.txt by typing echo "I love getting my OS to work!" > cat-input.txt Then run ./kos -a "/cs/faculty/rich/cs170/test_execs/cat" < cat-input.txt Cool, yes? Finally, write your own test codes based on the ones in /cs/faculty/rich/cs170/test_execs. You probably can't do much because the only system calls your OS supports at this point are read(), write(), and exit(). If you write a test code, and nothing happens, run with the -d e flag and look for "Unknown system call." That means your test code is (perhaps in a library) is making some system call that is not read(), write(), or exit(). However you can write test codes that make these three calls in lots of different orders. Step 20 -- Rest. It has been a long day.