Projects / Programming Assignments - CSCI 402, Spring 2013, TuTh Section

At the beginning of the semester, you are given a skeleton of an operating system with some of the lower-level machine specific parts filled in. In stages, during the course of the semester you are asked to fill in important missing pieces of various subsystems in the OS. The OS is based on the x86 architecture and should, in theory, compile to an image that can be run on a bare x86 machine. For simplicity, however, you will be testing and developing parts of the OS on an x86 emulator called Qemu.

The OS is written in C, so the programming assignments listed below assume familiarity with the C language. If you have not programmed in C before, we encourage you to pick it up on your own as quickly as possible before the semester starts.

You are allowed to do the kernel assignments in teams of 2 to 4 students [BC: updated 1/19/2013]. Please read carefully the section below on constituting teams.

Warmup Assignments:

• Warmup Assignment 1 (doubly-linked circular list in C), due 11:45PM, 2/1/2013. Electronic submissions only.
• Warmup Assignment 2 (token bucket emulation in C), due 11:45PM, 2/22/2013. Electronic submissions only.

• Kernel Assignment 1 (processes and threads), due 11:45PM, 3/24/2013. Electronic submissions only.
• Kernel Assignment 2 (VFS layer), due 11:45PM, 4/14/2013. Electronic submissions only.
• Kernel Assignment 3 (virtual memory), due 11:45PM, 5/3/2013. Electronic submissions only (this is an extra credit assignment).

Warmup 1:		10%
Warmup 2:		20%
Kernel 1:		35%
Kernel 2:		35%
Kernel 3:		25%	(this is an extra credit assignment)

The warmup assignments must run on nunki.usc.edu.

We would like to encourage that everyone contributed equally to the team (or at least for you to declare as so in your README file). Therefore, your scores are maximized when you have an equal-share split. There may be penalty for not dividing the scores equally. The actual equations we will you to determine the scores is a bit involved. The basic idea is to use "standard deviation" to measure how skew the amount of sharing is going on. When you have an equal-share split, the standard deviation is zero, that corresponds to an upper bound (i.e., no penalty). When you have a completely skewed distribution (e.g., 0/0/100 split), you would get the highest standard deviation and the most penalty. To make things easier to figure out, we have provided a program on nunki.usc.edu/aludra.usc.edu for you to run. The syntax is:

    ~csci551b/bin/cs402calc grade p1 p2 ...

    ~csci551b/bin/cs402calc 80 20 30 50

If the grader gave 80 points and the breakdown is 0/50/50%, you should run:

    ~csci551b/bin/cs402calc 80 0 50 50

If the grader gave 80 points and the breakdown is 0/0/100%, you should run:

    ~csci551b/bin/cs402calc 80 0 0 100

It's totally up to your team to decide how you want to assign the splits. If there are disagreements, the instructor will have to interview the team members (and ask questions such as, "What do this code do?") and make a decision.

Here is how teaming will work:

• You must decide on your project team for doing the kernel assignments before the end of day on 2/24/2013 (two days after warmup 2 is due). Each team member should send the instructor an email listing their team members (and copy all other team members on this e-mail).
• Once the team is formed and the kernel assignments started, the only way you can change your team is by mutually agreed swaps with another team. Please keep the instructor informed if swaps are being done.
• All students not belonging to a group by the group forming deadline will be assigned a group by the instructor using a randomized algorithm (unless you have indicated that you want to do the kernel assignments by yourself) as follows:
  • form a random list consisting of these students (by a random drawing)
  • assign 3 to a group starting from the beginning of the list
  • remaining students join these randomly formed groups [BC: updated 1/19/2013]
  • boundary conditions? Let's hope we won't get there.
  We know that this algorithm is far from perfect, that's why it is specified before the semester starts. This way, you can plan how to form teams given that you know our algorithm for assigning teams.

• More is not necessarily better: having more people on the team also increases the coordination overhead.
• In many teams, some of the contributors start to assume a passive role, leaving others to assume the larger burden of the work. To avoid this, you should have frequent team meetings, carefully delineated programming responsibilities, and internal deadlines that help ensure that everyone does their part on time.
• It's very risky to work with someone you don't know. You are strongly encourage to "work with" other students when you are doing the warmup assignments. "Work with" means to work together at a high level (i.e., not coding). When it comes time to write code for the warmup assignments, you must do it completely independently. This way, you can learn the habits of potential teammates.

The following was by Pradeep Nayak, who took CS 402 in Fall 2012.

---------- Forwarded message ----------
From: Pradeep Nayak
Date: Thu, Oct 18, 2012 at 11:55 AM
Subject: [usc-cs402-f12: 822] Using Dropbox to manage and backup your code!
To: usc-cs402-f12@googlegroups.com

As professor mentioned in class, it's important to back up your code.I already use Dropbox for a lot of other things, hence using that as an example here. You can as well use Ubuntu One and the process should be the same. If you don't have a dropbox account, you can create one by using this link : http://db.tt/oLKgb3hw :-)

We will be using Dropbox to version control your code with git and also backing it up on Dropbox.

Things you need installed before you start:

1. You need git first! A simple way to install this would be sudo apt-get install git
2. Setup Dropbox for Ubuntu:https://www.dropbox.com/install?os=lnx. If you followed the default install procedure i.e (I agree to license terms -> Next ->.. Finish) you would be able to see a folder called Dropbox under /home//Dropbox where username is the current user you are logged in to the machine

Lets get started:

1. Set up a bare central repository on Dropbox. This is the place where you would be backing up your code along with version control

       cd ~/Dropbox
       mkdir kernel-project.git
       cd kernel-project.git
       git init --bare

2. Clone your remote repository on to your home directory. This is where you would be working on your project, committing changes etc.

       cd ~
       git clone ~/Dropbox/kernel-project.git

   You would now see a folder called kernel-project in your home directory.

3. Lets make the first commit:

       cd ~/kernel-project
       echo "Hello World" > foo.txt
       git add .
       git commit . -m "First Commit"

   Now you have committed the changes to your local repository. Next step is to push your code to Dropbox

4. Push code to Dropbox. Make sure you are in the kernel-project folder

       git push origin master

   So if your Dropbox is already running in background it would update your changes to Dropbox cloud. If not, the next time open Dropbox it would sync your changes on to the Dropbox servers.

Also, git is awesome to use when you are working on a collaborative project like this.

Useful resources:

• git cheat sheet
• Introduction to Git by Scott Chacon video
• Pro Git

1. The class programming assignments will be C code to be developed on a UNIX environment. No other programming language will be accepted and your program must compile and run with a Makefile as is. You must be familiar with the UNIX development environment (vi/pico/emacs, cc/gcc, make, etc.) Please read the programming FAQ if you need a refresher on C file I/O and bit/byte manipulications.

2. For the warmup assignments, you should use your USC accounts and preferably work on the Solaris machines via ssh for testing. The final (submitted) program must run on nunki.usc.edu because we are going to test it in that environment. Please note that we can only grade your submission from the grading account on nunki.usc.edu. It would be a good idea to test your program in another student's account to make sure that it runs everywhere. By the way, you should not do the whole program development there, as nunki is a general purpose server - under heavy use by many students. But you should definitely test your program there. Please also note that regrades can only be done from the grading account on nunki.usc.edu.

3. For the warmup assignments, please do not hardcode any directory path in your code! If you hardcode something like "/home/scf-..." or "/auto/home/scf-..." in your code to access something in your home directory and the grader cannot access these directories during grading, we will not be able to make changes to the overall filesystem on nunki. You may only get a score of 1 point as a result of this. So, please make sure you are not doing this. All path should be specified externally, as far as your code is concerned.

   The only path that you can hardcode is probably "/tmp", and even that is not a great idea. What you can do is to define such a path as a compile time variable and pass it to your program. For example, you can use the following to define TMPDIR to be equal to "/tmp":

       gcc ... -DTMPDIR=\"/tmp\" ...

   Then in your code, you can do:

       char tmpfile[256];
   
       snprintf(tmpfile, sizeof(tmpfile), "%s/XXXXXX", TMPDIR);
       ... mkstemp(tmpfile) ...

   Basically, using a compile time variable is the same as doing the following in your code:

       #define TMPDIR "/tmp"

   The difference is that you are doing it outside of your code, and this is cleaner.

4. We will make grading guidelines available at least one week before an assignment is due. We will grade based on the grading guidelines (may be with minor adjustments). The general rule is that you do not get credit for simply coding. You only get credit for getting your code to work correctly according to the spec. (Please do not ask the grader to look at your code so you can get more points because you have done a lot of coding and your code looks like it should work. If your code is close to working, it's your responsiblity to get your code to work.)

5. Late submissions will receive severe penalties. Please see the late policy.

6. All submissions will be timestamped by the submission server and receipts (known as tickets) will be issued. Whether your submission arrived to the server by the deadline is determined by the timestamp. Please do not delete your receipts/tickets from your home directory on nunki.usc.edu.

7. If you sign up late for this class, you are still required to turn in all the programming assignments on time or you will receive a score of zero for the applicable assignments. No exceptions! This requirement also applys to students on the wait list.

8. You must follow the Electronic Submission Guidelines when you submit programming assignments. Please note that this is a fairly new procedure and very different from previous procedures. Please make sure you read the output of the bsubmit program carefully. It should look similar to the sample output given on the bsubmit page. The timestamped upload ticket issued by the Bistro server is a proof that the server has received your submission (and you do not need additional proof such as an e-mail confirmation). You should also verify what you have submitted is what you intended to submit by following the Verify Your Submission procedure. Please note that it is your responsibility to ensure that you have submitted valid submissions and that you have received timestampted upload tickets for them.

For Warmup Assignment #2, if you use code fragments obtained from public sources (or code derived or adapted from them), you must cite such code fragments explicitly to give credits properly. Just mention that you got some code in your README file is no good. You must surround such source code by the following comment blocks:

    /*
     * Begin code I did not write.
     * The code was downloaded from <URL>.
     * If the source code requires you to include copyright, put copyright here.
     */
    [ code you did not write yourself ]
    /*
     * End code I did not write.
     */

There is one exception to the rule for using code you did not write... Code given to the entire class in a class taken by you is permitted, provided that you cite such code properly. You must surround such source code by the following comment blocks:

    /*
     * Begin code I did not write.
     * The code was given to the (class information) in
     *     (semester, year) at (institution name).
     * If the source code requires you to include copyright, put copyright here.
     */
    [ code you did not write yourself ]
    /*
     * End code I did not write.
     */

One line (128 characters max) of change is defined as one of the following:

• Add 1 line before (or after) line x of file y
• Delete line x of file y
• Replace line x of file y by 1 line

Afterwards, additional modifications cost 12 points per line until 7 days past the submission deadline. After 7 days past the submission deadline, an additional modification costs 30 points per line.

Please note that this applies to source code, Makefile, and README files.

Chances are that you have corrupted memory. This usually means that you have corrupted memory a while back. It just happened that when you call free(), the corrupted memory caused a bus error or the execution of an illegal instruction. Bus errors and illegal instructions are basically the same thing as segmentation faults.

How does one corrupt memory? You can write beyond an allocated memory block. You can free the same object twice. You can free an object that was not allocated. You can write to an object that's already freed. These bugs are hard to find because most of the them you only see that there is problem long time after you have corrupted memory. If you have access to a professional/expensive debugging tool, it may be helpful. Otherwise, you just need to do binary search and see where the bug(s) might be. There's no magical cures in debugging memory corruption bugs, not even for professionals! I, unfortunately, do not have any magic tricks that can help anyone find memory corruption bugs.

One thing you might try is to temporarily turn off memory deallocation (if you suspect that you have freed the same object twice or freed an object that was not allocated). You can do the following to define free() as a no-op in a common header file when you are debugging:

    #ifdef DEBUGGING_MEMORY_CORRUPTION
    #ifdef free
    #undef free
    #endif /* free */
    #define free
    #endif /* DEBUGGING_MEMORY_CORRUPTION */

As your code gets more and more complicated, you may get more of these bugs. This is one reason why you want to keep your code nice and clean.

If you develop your code on a Linux machine, you can try valgrind. I have not tried this tool and I do not know if it's suitable for our programming assignments. But I've heard good things (e.g., easy to use) and bad things about it (e.g., it only runs on Linux). Please try it at your own risk. Also, if you want to give it a try, incorporate it in your programs as early as possible. Don't start learning it when bugs are happening and deadlines are approaching.