Return-Path: william@bourbon.usc.edu Delivery-Date: Wed May 14 10:33:27 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m4EHXRLV032237 for ; Wed, 14 May 2008 10:33:27 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m4EH8d7D025904 for ; Wed, 14 May 2008 10:08:39 -0700 Message-Id: <200805141708.m4EH8d7D025904@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Regarding Final Exam Date: Wed, 14 May 2008 10:08:39 -0700 From: Bill Cheng Someone wrote: > I would like to know if calculators are allowed as it would > help in doing calculations faster for problems like fair queuing. Calculators will *not* be allowed. You will have to do calculations by hand. So, please calculate carefully and check your answers twice. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat May 10 16:20:35 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m4ANKZdf032068 for ; Sat, 10 May 2008 16:20:35 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m4AMuIj4020368 for ; Sat, 10 May 2008 15:56:18 -0700 Message-Id: <200805102256.m4AMuIj4020368@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: token bucket Date: Sat, 10 May 2008 15:56:18 -0700 From: Bill Cheng Someone wrote: > In the class you said that you will post a question on token > bucket in the extra homework. I would be obliged if you could do > so. It's been there since last Monday! Please see problem 4 at: http://merlot.usc.edu/cs551-s08/homeworks/hw-extra.html May be you need to clear your browser cache! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri May 9 19:57:18 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m4A2vINt020955 for ; Fri, 9 May 2008 19:57:18 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m4A2X8g4002446 for ; Fri, 9 May 2008 19:33:08 -0700 Message-Id: <200805100233.m4A2X8g4002446@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: From CS551 grader Date: Fri, 09 May 2008 19:33:08 -0700 From: Bill Cheng Hi everyone, Please remember that you only have to *request* a regrade soon. It does not have to be *resolved* within a week. Please give the grader some breathing room at this time. Thanks for your consideration! -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- Date: Fri, 09 May 2008 17:14:31 -0700 From: Hideaki Katagishi To: cs551@merlot.usc.edu Subject: From CS551 grader Hi class, This is Hideaki, CS551 grader. I have received some regrading requests from students, but I can't respond emails until next Mon. 2pm since I'm also having finals. I understand you are really worried about your grade, but we can resolve it during office hour or exchanging emails later. If I don't respond your emails soon, please be patient. I will reply back for sure. Thank you for your understanding. Sincerely, Hideaki Katagishi Return-Path: william@bourbon.usc.edu Delivery-Date: Wed May 7 13:03:22 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m47K3Mua014435 for ; Wed, 7 May 2008 13:03:22 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m47JdUtB010171 for ; Wed, 7 May 2008 12:39:30 -0700 Message-Id: <200805071939.m47JdUtB010171@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: slide 31 of lecture 14 on 3/4/2008 ??? Date: Wed, 07 May 2008 12:39:30 -0700 From: Bill Cheng Someone wrote: > As is said, final material starts from slide 31 of lecture 14 on > 3/4/2008. But on class > website, lecture 14 only has 30 slides, there's no slide 31. Is there > anything wrong? Oops! It should be slide 1 of lecture 15 on 3/6/08. It's fixed now. Thanks for catching it! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue May 6 10:14:09 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m46HDpLf029503; Tue, 6 May 2008 10:13:51 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m46Go8SO007460; Tue, 6 May 2008 09:50:08 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m46Go8ms007459; Tue, 6 May 2008 09:50:08 -0700 Date: Tue, 6 May 2008 09:50:08 -0700 From: Bill Cheng Message-Id: <200805061650.m46Go8ms007459@bourbon.usc.edu> To: cs530@merlot.usc.edu, cs551@merlot.usc.edu, cs558@merlot.usc.edu Subject: Office hour today... Hi everyone, I'm on campus today. So, I'll hold office hour between 1:30pm and 3:30pm. If you already have an appointment with me, please come at your appointment time. If you cannot make it during this time, please send e-mail to me to make an appointment. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun May 4 23:39:38 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m456dcrq003339 for ; Sun, 4 May 2008 23:39:38 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m456G7l3030861 for ; Sun, 4 May 2008 23:16:07 -0700 Message-Id: <200805050616.m456G7l3030861@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS 551 Question Date: Sun, 04 May 2008 23:16:07 -0700 From: Bill Cheng Someone wrote: > Why in your final topic summary, all multicast papers are missing? Oops! Fixed! Thanks for letting me know! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu May 1 16:50:08 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m41No8En008916 for ; Thu, 1 May 2008 16:50:08 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m41NR4n8008473 for ; Thu, 1 May 2008 16:27:04 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m41NR4F0008472 for cs551@merlot; Thu, 1 May 2008 16:27:04 -0700 Date: Thu, 1 May 2008 16:27:04 -0700 From: Bill Cheng Message-Id: <200805012327.m41NR4F0008472@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Fina exam topics... Hi everyone, I've posted a summary of the topics covered (not all topics covered are listed) at the class web page. Please see the "news" (red) section of the class web page. The final exam will be closed book, closed notes, and closed everything (and no "cheat sheet"). Also, no calculators, cell phones, or any electronic gadgets are allowed. Please bring a photo ID. Your ID will be collected at the beginning of the exam and will be returned to you when you turn in your exam. There will be assigned seating. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 25 15:06:31 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3PM6Vve004389 for ; Fri, 25 Apr 2008 15:06:31 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3PLiH5I006905 for ; Fri, 25 Apr 2008 14:44:17 -0700 Message-Id: <200804252144.m3PLiH5I006905@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: High load nunki problems Date: Fri, 25 Apr 2008 14:44:17 -0700 From: Bill Cheng Someone wrote: > As allready mentioned by many students, the load on nunki is very > high and it is getting really tough to do any testing or > debugging. Process like "sim-outorder" are taking about 20% CPU > on an average and at any time there are atleast 4-6 such > processes. This is is giving us a very hard time doing any > testing. All righty! I guess this is unusual enough, so I'm extending the deadline for final project part (2) by 1 day. If the load on nunki is still high tomorrow, there will be no more extensions! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 25 13:13:21 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3PKDLEY026234 for ; Fri, 25 Apr 2008 13:13:21 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3PJp7Ap004039 for ; Fri, 25 Apr 2008 12:51:07 -0700 Message-Id: <200804251951.m3PJp7Ap004039@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Fri, 25 Apr 2008 12:51:07 -0700 From: Bill Cheng Someone wrote: > there are multiple files that are to be run in the script . So i guess it > will done in sometime. I dont believe it will be in this condition after > today.Looks like many people have done this at the same time. that is > causing the problems. Let's hope the load will go down soon enough. -- Bill Cheng // bill.cheng@usc.edu On Fri, Apr 25, 2008 at 12:00 PM, Bill Cheng wrote: > You mean there is no submission for EE 557 this week? So, > we should expect the load to stay this high all the way > to 5/9? > > Many different users are running "sim-outorder". So, it's > not just one student. > -- > Bill Cheng // bill.cheng@usc.edu > > > > -----Original Message----- > Date: Fri, 25 Apr 2008 11:52:57 -0700 > To: Bill Cheng > Subject: Re: nunki problems > > Hey, > The weekend submissions have been scrapped. It might have been one off > person who seems to be running the test. > > On Fri, Apr 25, 2008 at 11:23 AM, Bill Cheng > wrote: > > > Someone wrote: > > > > > it is due on 9th of may, but there is a submission of report every > week. > > > > What about the deadline this week? I'm trying to figure out > > if there is any point in extending the deadline for one day! > > (If the deadline for them is next Monday, then there is no > > point for the extension.) > > -- > > Bill Cheng // bill.cheng@usc.edu http://merlot.usc.edu/william/usc/> > > > > > > > > > > On Fri, Apr 25, 2008 at 11:10 AM, Bill Cheng > > wrote: > > > Someone wrote: > > > > > > > Even Aludra is taking the same time. I tried that option as well > > > > > > Looks like the same process, "sim-outorder", is taking up > > > all the CPU cycles. > > > > > > By the way, does anyone know when the EE 557 project is due? > > > -- > > > Bill Cheng // bill.cheng@usc.edu > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > On Fri, Apr 25, 2008 at 10:54 AM, Bill Cheng < > william@bourbon.usc.edu> > > > wrote: > > > > > > > Someone wrote: > > > > > > > > > Yesterday I tried working on an ITS SUN machine in SAL, my > > > application > > > > would > > > > > start, and immediately before it gives me my command prompt, it > > will > > > > > shutdown with no errors. I did not want to spend the time > > debugging > > > the > > > > > issue since I do not know if there is some limitation on the > > machine > > > and > > > > I > > > > > would just be wasting my time, and also, if I am able to fix it > > for > > > the > > > > ITS > > > > > machine, it might not work on nunki anymore, that is why I > > continued > > > > working > > > > > on nunki despite its issues. > > > > > > > > Hmm.. I do not know if this something with your code or it's > > > > with the ITS SUN machines. I have used ITS SUN machines before > > > > and had no problem and I have not heard anyone complained about > > > > the ITS SUN machines! > > > > > > > > By the way, if compiling is taking too long, you can compile > > > > on aludra! You can not run multiple processes on aludra. But > > > > compiling should be fine. > > > > -- > > > > Bill Cheng // bill.cheng@usc.edu > > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > > > On Fri, Apr 25, 2008 at 10:18 AM, Bill Cheng < > > william@bourbon.usc.edu> > > > > wrote: > > > > > > > > > Someone wrote: > > > > > > > > > > > Nunki was practically not working yesterday. > It > > was > > > > taking > > > > > > 10-15 minutes to compile the code. Not only there are cs551 > > > projects > > > > and > > > > > > cs402 projects but also graphically heavy ee577 projects > going > > on. > > > > > > > > > > Nunki is a shared machine and we have no control over it. > > > > > As I mentioned during lectures, towards the end of the > > > > > semesters, things can get crazy. Some classes will even be > > > > > installing and running large systems (such as databases) on > > > > > nunki and eat up all the system resources. Also, as you can > > > > > see, some students in our class are not making things better. > > > > > > > > > > One solution is to work on an ITS workstation in SAL today! > > > > > -- > > > > > Bill Cheng // bill.cheng@usc.edu > > > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 24, 2008 at 11:18 AM, Bill Cheng < > > > william@bourbon.usc.edu > > > > > > > > > > wrote: > > > > > > Someone wrote: > > > > > > > > > > > > > Well, if the grader is grading the assignment after > everyone > > > has > > > > > > > submitted their assignments (nunki is quiet), then > sv_node > > will > > > > > > > consume the top places on the CPU usage because it's the > > only > > > > thing > > > > > > > running. This doesn't mean it's busy waiting. Certificate > > > > generation > > > > > > > (I've noticed) is CPU-intensive, but a necessary step to > > > complete > > > > the > > > > > > > project. I think such a guideline could be construed to > > > unfairly > > > > > > > reduce points on a project that doesn't busy wait. > > > > > > > > > > > > If your sv_node is giving up the processor, then it should > > > > > > not show up near the top. If your job sleeps for 100ms, > > > > > > after it wakes up, it polls (which take microseconds), and > > > > > > goes right back to sleep. It should only use a tiny > fraction > > > > > > of the CPU! You can try the following yourself. Write a > > > > > > small program that does the following: > > > > > > > > > > > > for (i=0; i < 500; i++) { > > > > > > usleep(100000); > > > > > > } > > > > > > > > > > > > run it and do "top" at the same time. This job should not > > > > > > show up at all! > > > > > > > > > > > > You are correct that certificate generation can take some > > > > > > time, that's why the grading guidelines says to wait 5 > > > > > > seconds. I've just change the busy-wait to the following: > > > > > > > > > > > > in 4th window, wait 5 seconds, type: > > > > > > top > > > > > > (watch this for 10 seconds and press "q" or > > > > > > > > > to > > > > > > exit "top") > > > > > > (if sv_node is taking up the top spots in > CPU > > > > > percentages > > > > > > and show high percentages, if so, this is > > > considered > > > > > > busy-wait and should take points off later, > > > please > > > > see > > > > > > busy-wait in the negative points section) > > > > > > > > > > > > > One suggestion to detect this is to look for non-blocking > > IO. > > > > Maybe > > > > > > > look for the setsockopt that places it in the > non-blocking > > > mode. > > > > > > > > > > > > I think if you are eating up CPU cycles, you are doing some > > > > > > sort of a busy-wait. If it's a strange bug, you should be > > > > > > able to insert calls to usleep() to fix the problem! > > > > > > > > > > > > Finally, if the grader mistaken your code for busy-wait, you > > > > > > can resolve it during regrading. So, if your code is not > > > > > > staying in a tight loop waiting for some flags to change > > > > > > state, there is nothing to worry about! > > > > > > -- > > > > > > Bill Cheng // bill.cheng@usc.edu > > > > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Apr 24, 2008, at 10:10 , Bill Cheng wrote: > > > > > > > Someone wrote: > > > > > > > > > > > > > >> I'm trying to test on nunki now and have just seen the > load > > > > average > > > > > > >> rise to and stay at for about 10 minutes 69.2. Since > nunki > > > has 4 > > > > > > >> processors, this means that nunki is over 17 times over > > it's > > > > optimal > > > > > > >> load limit. It takes 20 minutes to build my project from > > > scratch. > > > > > > >> > > > > > > >> This is making testing very difficult. Just letting you > > know > > > > about > > > > > > >> nunki's status ahead of time. > > > > > > > > > > > > > > Unfortunately, nunki is a shared machine and we cannot > > control > > > > > > > other user. It may be best to work on an ITS workstation > > (I'm > > > > > > > not sure if you can just ssh to one of them). *If* nunki > > goes > > > > > > > down for over 2 hours between now and the project due > time, > > > > > > > please let me know and I will extend the deadline by one > > day. > > > > > > > > > > > > > > I also see that sv_node were sometimes eating up the CPU > > time > > > > > > > on nunki. So, I'm increasing the "busy-wait" penalty to > 30 > > > > > > > points and added a test in (A.2.a) for it: > > > > > > > > > > > > > > in 4th window, wait 5 seconds, type: > > > > > > > top > > > > > > > (see if sv_node is taking up the top spots in CPU > > > > > > > percentages, if so, this is considered busy-wait > > > > > > > and should take points off later, please see > > busy-wait > > > > > > > in the negative points section) > > > > > > > (press to kill the "top" command or > press > > "q") > > > > > > > > > > > > > > ... > > > > > > > > > > > > > > Using busy-wait (with non-blocking sockets or > > non-blocking > > > I/O) > > > > : > > > > > > > -30 points > > > > > > > > > > > > > > So, this is a easy way to check if you are doing > qualifies > > > > > > > as doing busy-wait (i.e., eating up all the CPU cycles on > > > > > > > nunki). If you are doing something like this, the fix is > > > > > > > simple! Right before you poll, sleep for 100 millisecond > > > > > > > before doing the poll. If that's too slow, sleep for 50 > > > > > > > millisecond. > > > > > > > -- > > > > > > > Bill Cheng // bill.cheng@usc.edu > > > http://merlot.usc.edu/william/usc/> Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 25 12:20:27 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3PJKQBb021531 for ; Fri, 25 Apr 2008 12:20:26 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3PIwD6N002570 for ; Fri, 25 Apr 2008 11:58:13 -0700 Message-Id: <200804251858.m3PIwD6N002570@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Fri, 25 Apr 2008 11:58:13 -0700 From: Bill Cheng Someone wrote: > You are correct, I had to face this issue due to last minute work even > though there is no portability issue, I am developing it for Nunki :( I dont > know about other groups, but it would help us run all the test cases > properly if there was an extension. Thanks ! What I was saying is that you should not leave things to the last minute! (But I understand it's too late for this.) -- Bill Cheng // bill.cheng@usc.edu On Fri, Apr 25, 2008 at 11:23 AM, Bill Cheng wrote: > Someone wrote: > > > it is due on 9th of may, but there is a submission of report every week. > > What about the deadline this week? I'm trying to figure out > if there is any point in extending the deadline for one day! > (If the deadline for them is next Monday, then there is no > point for the extension.) > -- > Bill Cheng // bill.cheng@usc.edu > > > > > On Fri, Apr 25, 2008 at 11:10 AM, Bill Cheng > wrote: > > Someone wrote: > > > > > Even Aludra is taking the same time. I tried that option as well > > > > Looks like the same process, "sim-outorder", is taking up > > all the CPU cycles. > > > > By the way, does anyone know when the EE 557 project is due? > > -- > > Bill Cheng // bill.cheng@usc.edu http://merlot.usc.edu/william/usc/> > > > > > > > > > > On Fri, Apr 25, 2008 at 10:54 AM, Bill Cheng > > wrote: > > > > > Someone wrote: > > > > > > > Yesterday I tried working on an ITS SUN machine in SAL, my > > application > > > would > > > > start, and immediately before it gives me my command prompt, it > will > > > > shutdown with no errors. I did not want to spend the time > debugging > > the > > > > issue since I do not know if there is some limitation on the > machine > > and > > > I > > > > would just be wasting my time, and also, if I am able to fix it > for > > the > > > ITS > > > > machine, it might not work on nunki anymore, that is why I > continued > > > working > > > > on nunki despite its issues. > > > > > > Hmm.. I do not know if this something with your code or it's > > > with the ITS SUN machines. I have used ITS SUN machines before > > > and had no problem and I have not heard anyone complained about > > > the ITS SUN machines! > > > > > > By the way, if compiling is taking too long, you can compile > > > on aludra! You can not run multiple processes on aludra. But > > > compiling should be fine. > > > -- > > > Bill Cheng // bill.cheng@usc.edu > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > On Fri, Apr 25, 2008 at 10:18 AM, Bill Cheng < > william@bourbon.usc.edu> > > > wrote: > > > > > > > Someone wrote: > > > > > > > > > Nunki was practically not working yesterday. It > was > > > taking > > > > > 10-15 minutes to compile the code. Not only there are cs551 > > projects > > > and > > > > > cs402 projects but also graphically heavy ee577 projects going > on. > > > > > > > > Nunki is a shared machine and we have no control over it. > > > > As I mentioned during lectures, towards the end of the > > > > semesters, things can get crazy. Some classes will even be > > > > installing and running large systems (such as databases) on > > > > nunki and eat up all the system resources. Also, as you can > > > > see, some students in our class are not making things better. > > > > > > > > One solution is to work on an ITS workstation in SAL today! > > > > -- > > > > Bill Cheng // bill.cheng@usc.edu > > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > > > On Thu, Apr 24, 2008 at 11:18 AM, Bill Cheng < > > william@bourbon.usc.edu > > > > > > > > wrote: > > > > > Someone wrote: > > > > > > > > > > > Well, if the grader is grading the assignment after everyone > > has > > > > > > submitted their assignments (nunki is quiet), then sv_node > will > > > > > > consume the top places on the CPU usage because it's the > only > > > thing > > > > > > running. This doesn't mean it's busy waiting. Certificate > > > generation > > > > > > (I've noticed) is CPU-intensive, but a necessary step to > > complete > > > the > > > > > > project. I think such a guideline could be construed to > > unfairly > > > > > > reduce points on a project that doesn't busy wait. > > > > > > > > > > If your sv_node is giving up the processor, then it should > > > > > not show up near the top. If your job sleeps for 100ms, > > > > > after it wakes up, it polls (which take microseconds), and > > > > > goes right back to sleep. It should only use a tiny fraction > > > > > of the CPU! You can try the following yourself. Write a > > > > > small program that does the following: > > > > > > > > > > for (i=0; i < 500; i++) { > > > > > usleep(100000); > > > > > } > > > > > > > > > > run it and do "top" at the same time. This job should not > > > > > show up at all! > > > > > > > > > > You are correct that certificate generation can take some > > > > > time, that's why the grading guidelines says to wait 5 > > > > > seconds. I've just change the busy-wait to the following: > > > > > > > > > > in 4th window, wait 5 seconds, type: > > > > > top > > > > > (watch this for 10 seconds and press "q" or > > > > > > > to > > > > > exit "top") > > > > > (if sv_node is taking up the top spots in CPU > > > > percentages > > > > > and show high percentages, if so, this is > > considered > > > > > busy-wait and should take points off later, > > please > > > see > > > > > busy-wait in the negative points section) > > > > > > > > > > > One suggestion to detect this is to look for non-blocking > IO. > > > Maybe > > > > > > look for the setsockopt that places it in the non-blocking > > mode. > > > > > > > > > > I think if you are eating up CPU cycles, you are doing some > > > > > sort of a busy-wait. If it's a strange bug, you should be > > > > > able to insert calls to usleep() to fix the problem! > > > > > > > > > > Finally, if the grader mistaken your code for busy-wait, you > > > > > can resolve it during regrading. So, if your code is not > > > > > staying in a tight loop waiting for some flags to change > > > > > state, there is nothing to worry about! > > > > > -- > > > > > Bill Cheng // bill.cheng@usc.edu > > > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Apr 24, 2008, at 10:10 , Bill Cheng wrote: > > > > > > Someone wrote: > > > > > > > > > > > >> I'm trying to test on nunki now and have just seen the load > > > average > > > > > >> rise to and stay at for about 10 minutes 69.2. Since nunki > > has 4 > > > > > >> processors, this means that nunki is over 17 times over > it's > > > optimal > > > > > >> load limit. It takes 20 minutes to build my project from > > scratch. > > > > > >> > > > > > >> This is making testing very difficult. Just letting you > know > > > about > > > > > >> nunki's status ahead of time. > > > > > > > > > > > > Unfortunately, nunki is a shared machine and we cannot > control > > > > > > other user. It may be best to work on an ITS workstation > (I'm > > > > > > not sure if you can just ssh to one of them). *If* nunki > goes > > > > > > down for over 2 hours between now and the project due time, > > > > > > please let me know and I will extend the deadline by one > day. > > > > > > > > > > > > I also see that sv_node were sometimes eating up the CPU > time > > > > > > on nunki. So, I'm increasing the "busy-wait" penalty to 30 > > > > > > points and added a test in (A.2.a) for it: > > > > > > > > > > > > in 4th window, wait 5 seconds, type: > > > > > > top > > > > > > (see if sv_node is taking up the top spots in CPU > > > > > > percentages, if so, this is considered busy-wait > > > > > > and should take points off later, please see > busy-wait > > > > > > in the negative points section) > > > > > > (press to kill the "top" command or press > "q") > > > > > > > > > > > > ... > > > > > > > > > > > > Using busy-wait (with non-blocking sockets or > non-blocking > > I/O) > > > : > > > > > > -30 points > > > > > > > > > > > > So, this is a easy way to check if you are doing qualifies > > > > > > as doing busy-wait (i.e., eating up all the CPU cycles on > > > > > > nunki). If you are doing something like this, the fix is > > > > > > simple! Right before you poll, sleep for 100 millisecond > > > > > > before doing the poll. If that's too slow, sleep for 50 > > > > > > millisecond. > > > > > > -- > > > > > > Bill Cheng // bill.cheng@usc.edu > > http://merlot.usc.edu/william/usc/> Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 25 12:18:08 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3PJI83C021321 for ; Fri, 25 Apr 2008 12:18:08 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3PIttLr002488 for ; Fri, 25 Apr 2008 11:55:55 -0700 Message-Id: <200804251855.m3PIttLr002488@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Fri, 25 Apr 2008 11:55:55 -0700 From: Bill Cheng Someone wrote: > I had tried doing the same on my Fedora core 6 machine... but then the > compilation there and on the sun machines was different It took me more time > to fix those compilation errors & runtime then to write the code. If you want your code to run on multiple platforms, you need to do it starting on day one and call all the commond functions (instead of the platform specific ones). Otherwise, it's a lot of work. -- Bill Cheng // bill.cheng@usc.edu On Fri, Apr 25, 2008 at 11:21 AM, Bill Cheng wrote: > Someone wrote: > > > Even I am finding it very difficult to upload and run my test cases on > the > > Nunki Server as it is taking a lot of time. I heard that the CSCI 402 > > submissions are due this weekend.. > > Looks like it's just us and EE 557 that are slowing down nunki > at this time! > > > Is there anything that could be done ? > > As I've mentioned many times during the semester, "start > early (and finish early)". And you should always leave room > for surprises. Other things you could have done is to write > platform independent code so you can debug on cygwin and > Linux machines; although you still need to make sure that it > runs exactly the same on Solaris. But I know it's too late > for all these. > -- > Bill Cheng // bill.cheng@usc.edu > > > > > On Fri, Apr 25, 2008 at 11:10 AM, Bill Cheng > wrote: > > Someone wrote: > > > > > Even Aludra is taking the same time. I tried that option as well > > > > Looks like the same process, "sim-outorder", is taking up > > all the CPU cycles. > > > > By the way, does anyone know when the EE 557 project is due? > > -- > > Bill Cheng // bill.cheng@usc.edu http://merlot.usc.edu/william/usc/> > > > > > > > > > > On Fri, Apr 25, 2008 at 10:54 AM, Bill Cheng > > wrote: > > > > > Someone wrote: > > > > > > > Yesterday I tried working on an ITS SUN machine in SAL, my > > application > > > would > > > > start, and immediately before it gives me my command prompt, it > will > > > > shutdown with no errors. I did not want to spend the time > debugging > > the > > > > issue since I do not know if there is some limitation on the > machine > > and > > > I > > > > would just be wasting my time, and also, if I am able to fix it > for > > the > > > ITS > > > > machine, it might not work on nunki anymore, that is why I > continued > > > working > > > > on nunki despite its issues. > > > > > > Hmm.. I do not know if this something with your code or it's > > > with the ITS SUN machines. I have used ITS SUN machines before > > > and had no problem and I have not heard anyone complained about > > > the ITS SUN machines! > > > > > > By the way, if compiling is taking too long, you can compile > > > on aludra! You can not run multiple processes on aludra. But > > > compiling should be fine. > > > -- > > > Bill Cheng // bill.cheng@usc.edu > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > On Fri, Apr 25, 2008 at 10:18 AM, Bill Cheng < > william@bourbon.usc.edu> > > > wrote: > > > > > > > Someone wrote: > > > > > > > > > Nunki was practically not working yesterday. It > was > > > taking > > > > > 10-15 minutes to compile the code. Not only there are cs551 > > projects > > > and > > > > > cs402 projects but also graphically heavy ee577 projects going > on. > > > > > > > > Nunki is a shared machine and we have no control over it. > > > > As I mentioned during lectures, towards the end of the > > > > semesters, things can get crazy. Some classes will even be > > > > installing and running large systems (such as databases) on > > > > nunki and eat up all the system resources. Also, as you can > > > > see, some students in our class are not making things better. > > > > > > > > One solution is to work on an ITS workstation in SAL today! > > > > -- > > > > Bill Cheng // bill.cheng@usc.edu > > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > > > On Thu, Apr 24, 2008 at 11:18 AM, Bill Cheng < > > william@bourbon.usc.edu > > > > > > > > wrote: > > > > > Someone wrote: > > > > > > > > > > > Well, if the grader is grading the assignment after everyone > > has > > > > > > submitted their assignments (nunki is quiet), then sv_node > will > > > > > > consume the top places on the CPU usage because it's the > only > > > thing > > > > > > running. This doesn't mean it's busy waiting. Certificate > > > generation > > > > > > (I've noticed) is CPU-intensive, but a necessary step to > > complete > > > the > > > > > > project. I think such a guideline could be construed to > > unfairly > > > > > > reduce points on a project that doesn't busy wait. > > > > > > > > > > If your sv_node is giving up the processor, then it should > > > > > not show up near the top. If your job sleeps for 100ms, > > > > > after it wakes up, it polls (which take microseconds), and > > > > > goes right back to sleep. It should only use a tiny fraction > > > > > of the CPU! You can try the following yourself. Write a > > > > > small program that does the following: > > > > > > > > > > for (i=0; i < 500; i++) { > > > > > usleep(100000); > > > > > } > > > > > > > > > > run it and do "top" at the same time. This job should not > > > > > show up at all! > > > > > > > > > > You are correct that certificate generation can take some > > > > > time, that's why the grading guidelines says to wait 5 > > > > > seconds. I've just change the busy-wait to the following: > > > > > > > > > > in 4th window, wait 5 seconds, type: > > > > > top > > > > > (watch this for 10 seconds and press "q" or > > > > > > > to > > > > > exit "top") > > > > > (if sv_node is taking up the top spots in CPU > > > > percentages > > > > > and show high percentages, if so, this is > > considered > > > > > busy-wait and should take points off later, > > please > > > see > > > > > busy-wait in the negative points section) > > > > > > > > > > > One suggestion to detect this is to look for non-blocking > IO. > > > Maybe > > > > > > look for the setsockopt that places it in the non-blocking > > mode. > > > > > > > > > > I think if you are eating up CPU cycles, you are doing some > > > > > sort of a busy-wait. If it's a strange bug, you should be > > > > > able to insert calls to usleep() to fix the problem! > > > > > > > > > > Finally, if the grader mistaken your code for busy-wait, you > > > > > can resolve it during regrading. So, if your code is not > > > > > staying in a tight loop waiting for some flags to change > > > > > state, there is nothing to worry about! > > > > > -- > > > > > Bill Cheng // bill.cheng@usc.edu > > > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Apr 24, 2008, at 10:10 , Bill Cheng wrote: > > > > > > Someone wrote: > > > > > > > > > > > >> I'm trying to test on nunki now and have just seen the load > > > average > > > > > >> rise to and stay at for about 10 minutes 69.2. Since nunki > > has 4 > > > > > >> processors, this means that nunki is over 17 times over > it's > > > optimal > > > > > >> load limit. It takes 20 minutes to build my project from > > scratch. > > > > > >> > > > > > >> This is making testing very difficult. Just letting you > know > > > about > > > > > >> nunki's status ahead of time. > > > > > > > > > > > > Unfortunately, nunki is a shared machine and we cannot > control > > > > > > other user. It may be best to work on an ITS workstation > (I'm > > > > > > not sure if you can just ssh to one of them). *If* nunki > goes > > > > > > down for over 2 hours between now and the project due time, > > > > > > please let me know and I will extend the deadline by one > day. > > > > > > > > > > > > I also see that sv_node were sometimes eating up the CPU > time > > > > > > on nunki. So, I'm increasing the "busy-wait" penalty to 30 > > > > > > points and added a test in (A.2.a) for it: > > > > > > > > > > > > in 4th window, wait 5 seconds, type: > > > > > > top > > > > > > (see if sv_node is taking up the top spots in CPU > > > > > > percentages, if so, this is considered busy-wait > > > > > > and should take points off later, please see > busy-wait > > > > > > in the negative points section) > > > > > > (press to kill the "top" command or press > "q") > > > > > > > > > > > > ... > > > > > > > > > > > > Using busy-wait (with non-blocking sockets or > non-blocking > > I/O) > > > : > > > > > > -30 points > > > > > > > > > > > > So, this is a easy way to check if you are doing qualifies > > > > > > as doing busy-wait (i.e., eating up all the CPU cycles on > > > > > > nunki). If you are doing something like this, the fix is > > > > > > simple! Right before you poll, sleep for 100 millisecond > > > > > > before doing the poll. If that's too slow, sleep for 50 > > > > > > millisecond. > > > > > > -- > > > > > > Bill Cheng // bill.cheng@usc.edu > > http://merlot.usc.edu/william/usc/> Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 25 11:45:45 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3PIjjVo018310 for ; Fri, 25 Apr 2008 11:45:45 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3PINVCc001794 for ; Fri, 25 Apr 2008 11:23:31 -0700 Message-Id: <200804251823.m3PINVCc001794@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Fri, 25 Apr 2008 11:23:31 -0700 From: Bill Cheng Someone wrote: > it is due on 9th of may, but there is a submission of report every week. What about the deadline this week? I'm trying to figure out if there is any point in extending the deadline for one day! (If the deadline for them is next Monday, then there is no point for the extension.) -- Bill Cheng // bill.cheng@usc.edu On Fri, Apr 25, 2008 at 11:10 AM, Bill Cheng wrote: > Someone wrote: > > > Even Aludra is taking the same time. I tried that option as well > > Looks like the same process, "sim-outorder", is taking up > all the CPU cycles. > > By the way, does anyone know when the EE 557 project is due? > -- > Bill Cheng // bill.cheng@usc.edu > > > > > On Fri, Apr 25, 2008 at 10:54 AM, Bill Cheng > wrote: > > > Someone wrote: > > > > > Yesterday I tried working on an ITS SUN machine in SAL, my > application > > would > > > start, and immediately before it gives me my command prompt, it will > > > shutdown with no errors. I did not want to spend the time debugging > the > > > issue since I do not know if there is some limitation on the machine > and > > I > > > would just be wasting my time, and also, if I am able to fix it for > the > > ITS > > > machine, it might not work on nunki anymore, that is why I continued > > working > > > on nunki despite its issues. > > > > Hmm.. I do not know if this something with your code or it's > > with the ITS SUN machines. I have used ITS SUN machines before > > and had no problem and I have not heard anyone complained about > > the ITS SUN machines! > > > > By the way, if compiling is taking too long, you can compile > > on aludra! You can not run multiple processes on aludra. But > > compiling should be fine. > > -- > > Bill Cheng // bill.cheng@usc.edu http://merlot.usc.edu/william/usc/> > > > > > > > > > > On Fri, Apr 25, 2008 at 10:18 AM, Bill Cheng > > wrote: > > > > > Someone wrote: > > > > > > > Nunki was practically not working yesterday. It was > > taking > > > > 10-15 minutes to compile the code. Not only there are cs551 > projects > > and > > > > cs402 projects but also graphically heavy ee577 projects going on. > > > > > > Nunki is a shared machine and we have no control over it. > > > As I mentioned during lectures, towards the end of the > > > semesters, things can get crazy. Some classes will even be > > > installing and running large systems (such as databases) on > > > nunki and eat up all the system resources. Also, as you can > > > see, some students in our class are not making things better. > > > > > > One solution is to work on an ITS workstation in SAL today! > > > -- > > > Bill Cheng // bill.cheng@usc.edu > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > On Thu, Apr 24, 2008 at 11:18 AM, Bill Cheng < > william@bourbon.usc.edu > > > > > > wrote: > > > > Someone wrote: > > > > > > > > > Well, if the grader is grading the assignment after everyone > has > > > > > submitted their assignments (nunki is quiet), then sv_node will > > > > > consume the top places on the CPU usage because it's the only > > thing > > > > > running. This doesn't mean it's busy waiting. Certificate > > generation > > > > > (I've noticed) is CPU-intensive, but a necessary step to > complete > > the > > > > > project. I think such a guideline could be construed to > unfairly > > > > > reduce points on a project that doesn't busy wait. > > > > > > > > If your sv_node is giving up the processor, then it should > > > > not show up near the top. If your job sleeps for 100ms, > > > > after it wakes up, it polls (which take microseconds), and > > > > goes right back to sleep. It should only use a tiny fraction > > > > of the CPU! You can try the following yourself. Write a > > > > small program that does the following: > > > > > > > > for (i=0; i < 500; i++) { > > > > usleep(100000); > > > > } > > > > > > > > run it and do "top" at the same time. This job should not > > > > show up at all! > > > > > > > > You are correct that certificate generation can take some > > > > time, that's why the grading guidelines says to wait 5 > > > > seconds. I've just change the busy-wait to the following: > > > > > > > > in 4th window, wait 5 seconds, type: > > > > top > > > > (watch this for 10 seconds and press "q" or > > > > > to > > > > exit "top") > > > > (if sv_node is taking up the top spots in CPU > > > percentages > > > > and show high percentages, if so, this is > considered > > > > busy-wait and should take points off later, > please > > see > > > > busy-wait in the negative points section) > > > > > > > > > One suggestion to detect this is to look for non-blocking IO. > > Maybe > > > > > look for the setsockopt that places it in the non-blocking > mode. > > > > > > > > I think if you are eating up CPU cycles, you are doing some > > > > sort of a busy-wait. If it's a strange bug, you should be > > > > able to insert calls to usleep() to fix the problem! > > > > > > > > Finally, if the grader mistaken your code for busy-wait, you > > > > can resolve it during regrading. So, if your code is not > > > > staying in a tight loop waiting for some flags to change > > > > state, there is nothing to worry about! > > > > -- > > > > Bill Cheng // bill.cheng@usc.edu > > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > > > > > > > On Apr 24, 2008, at 10:10 , Bill Cheng wrote: > > > > > Someone wrote: > > > > > > > > > >> I'm trying to test on nunki now and have just seen the load > > average > > > > >> rise to and stay at for about 10 minutes 69.2. Since nunki > has 4 > > > > >> processors, this means that nunki is over 17 times over it's > > optimal > > > > >> load limit. It takes 20 minutes to build my project from > scratch. > > > > >> > > > > >> This is making testing very difficult. Just letting you know > > about > > > > >> nunki's status ahead of time. > > > > > > > > > > Unfortunately, nunki is a shared machine and we cannot control > > > > > other user. It may be best to work on an ITS workstation (I'm > > > > > not sure if you can just ssh to one of them). *If* nunki goes > > > > > down for over 2 hours between now and the project due time, > > > > > please let me know and I will extend the deadline by one day. > > > > > > > > > > I also see that sv_node were sometimes eating up the CPU time > > > > > on nunki. So, I'm increasing the "busy-wait" penalty to 30 > > > > > points and added a test in (A.2.a) for it: > > > > > > > > > > in 4th window, wait 5 seconds, type: > > > > > top > > > > > (see if sv_node is taking up the top spots in CPU > > > > > percentages, if so, this is considered busy-wait > > > > > and should take points off later, please see busy-wait > > > > > in the negative points section) > > > > > (press to kill the "top" command or press "q") > > > > > > > > > > ... > > > > > > > > > > Using busy-wait (with non-blocking sockets or non-blocking > I/O) > > : > > > > > -30 points > > > > > > > > > > So, this is a easy way to check if you are doing qualifies > > > > > as doing busy-wait (i.e., eating up all the CPU cycles on > > > > > nunki). If you are doing something like this, the fix is > > > > > simple! Right before you poll, sleep for 100 millisecond > > > > > before doing the poll. If that's too slow, sleep for 50 > > > > > millisecond. > > > > > -- > > > > > Bill Cheng // bill.cheng@usc.edu > http://merlot.usc.edu/william/usc/> Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 25 11:43:53 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3PIhrWR018087 for ; Fri, 25 Apr 2008 11:43:53 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3PILdm9001739 for ; Fri, 25 Apr 2008 11:21:39 -0700 Message-Id: <200804251821.m3PILdm9001739@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Fri, 25 Apr 2008 11:21:39 -0700 From: Bill Cheng Someone wrote: > Even I am finding it very difficult to upload and run my test cases on the > Nunki Server as it is taking a lot of time. I heard that the CSCI 402 > submissions are due this weekend.. Looks like it's just us and EE 557 that are slowing down nunki at this time! > Is there anything that could be done ? As I've mentioned many times during the semester, "start early (and finish early)". And you should always leave room for surprises. Other things you could have done is to write platform independent code so you can debug on cygwin and Linux machines; although you still need to make sure that it runs exactly the same on Solaris. But I know it's too late for all these. -- Bill Cheng // bill.cheng@usc.edu On Fri, Apr 25, 2008 at 11:10 AM, Bill Cheng wrote: > Someone wrote: > > > Even Aludra is taking the same time. I tried that option as well > > Looks like the same process, "sim-outorder", is taking up > all the CPU cycles. > > By the way, does anyone know when the EE 557 project is due? > -- > Bill Cheng // bill.cheng@usc.edu > > > > > On Fri, Apr 25, 2008 at 10:54 AM, Bill Cheng > wrote: > > > Someone wrote: > > > > > Yesterday I tried working on an ITS SUN machine in SAL, my > application > > would > > > start, and immediately before it gives me my command prompt, it will > > > shutdown with no errors. I did not want to spend the time debugging > the > > > issue since I do not know if there is some limitation on the machine > and > > I > > > would just be wasting my time, and also, if I am able to fix it for > the > > ITS > > > machine, it might not work on nunki anymore, that is why I continued > > working > > > on nunki despite its issues. > > > > Hmm.. I do not know if this something with your code or it's > > with the ITS SUN machines. I have used ITS SUN machines before > > and had no problem and I have not heard anyone complained about > > the ITS SUN machines! > > > > By the way, if compiling is taking too long, you can compile > > on aludra! You can not run multiple processes on aludra. But > > compiling should be fine. > > -- > > Bill Cheng // bill.cheng@usc.edu http://merlot.usc.edu/william/usc/> > > > > > > > > > > On Fri, Apr 25, 2008 at 10:18 AM, Bill Cheng > > wrote: > > > > > Someone wrote: > > > > > > > Nunki was practically not working yesterday. It was > > taking > > > > 10-15 minutes to compile the code. Not only there are cs551 > projects > > and > > > > cs402 projects but also graphically heavy ee577 projects going on. > > > > > > Nunki is a shared machine and we have no control over it. > > > As I mentioned during lectures, towards the end of the > > > semesters, things can get crazy. Some classes will even be > > > installing and running large systems (such as databases) on > > > nunki and eat up all the system resources. Also, as you can > > > see, some students in our class are not making things better. > > > > > > One solution is to work on an ITS workstation in SAL today! > > > -- > > > Bill Cheng // bill.cheng@usc.edu > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > On Thu, Apr 24, 2008 at 11:18 AM, Bill Cheng < > william@bourbon.usc.edu > > > > > > wrote: > > > > Someone wrote: > > > > > > > > > Well, if the grader is grading the assignment after everyone > has > > > > > submitted their assignments (nunki is quiet), then sv_node will > > > > > consume the top places on the CPU usage because it's the only > > thing > > > > > running. This doesn't mean it's busy waiting. Certificate > > generation > > > > > (I've noticed) is CPU-intensive, but a necessary step to > complete > > the > > > > > project. I think such a guideline could be construed to > unfairly > > > > > reduce points on a project that doesn't busy wait. > > > > > > > > If your sv_node is giving up the processor, then it should > > > > not show up near the top. If your job sleeps for 100ms, > > > > after it wakes up, it polls (which take microseconds), and > > > > goes right back to sleep. It should only use a tiny fraction > > > > of the CPU! You can try the following yourself. Write a > > > > small program that does the following: > > > > > > > > for (i=0; i < 500; i++) { > > > > usleep(100000); > > > > } > > > > > > > > run it and do "top" at the same time. This job should not > > > > show up at all! > > > > > > > > You are correct that certificate generation can take some > > > > time, that's why the grading guidelines says to wait 5 > > > > seconds. I've just change the busy-wait to the following: > > > > > > > > in 4th window, wait 5 seconds, type: > > > > top > > > > (watch this for 10 seconds and press "q" or > > > > > to > > > > exit "top") > > > > (if sv_node is taking up the top spots in CPU > > > percentages > > > > and show high percentages, if so, this is > considered > > > > busy-wait and should take points off later, > please > > see > > > > busy-wait in the negative points section) > > > > > > > > > One suggestion to detect this is to look for non-blocking IO. > > Maybe > > > > > look for the setsockopt that places it in the non-blocking > mode. > > > > > > > > I think if you are eating up CPU cycles, you are doing some > > > > sort of a busy-wait. If it's a strange bug, you should be > > > > able to insert calls to usleep() to fix the problem! > > > > > > > > Finally, if the grader mistaken your code for busy-wait, you > > > > can resolve it during regrading. So, if your code is not > > > > staying in a tight loop waiting for some flags to change > > > > state, there is nothing to worry about! > > > > -- > > > > Bill Cheng // bill.cheng@usc.edu > > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > > > > > > > On Apr 24, 2008, at 10:10 , Bill Cheng wrote: > > > > > Someone wrote: > > > > > > > > > >> I'm trying to test on nunki now and have just seen the load > > average > > > > >> rise to and stay at for about 10 minutes 69.2. Since nunki > has 4 > > > > >> processors, this means that nunki is over 17 times over it's > > optimal > > > > >> load limit. It takes 20 minutes to build my project from > scratch. > > > > >> > > > > >> This is making testing very difficult. Just letting you know > > about > > > > >> nunki's status ahead of time. > > > > > > > > > > Unfortunately, nunki is a shared machine and we cannot control > > > > > other user. It may be best to work on an ITS workstation (I'm > > > > > not sure if you can just ssh to one of them). *If* nunki goes > > > > > down for over 2 hours between now and the project due time, > > > > > please let me know and I will extend the deadline by one day. > > > > > > > > > > I also see that sv_node were sometimes eating up the CPU time > > > > > on nunki. So, I'm increasing the "busy-wait" penalty to 30 > > > > > points and added a test in (A.2.a) for it: > > > > > > > > > > in 4th window, wait 5 seconds, type: > > > > > top > > > > > (see if sv_node is taking up the top spots in CPU > > > > > percentages, if so, this is considered busy-wait > > > > > and should take points off later, please see busy-wait > > > > > in the negative points section) > > > > > (press to kill the "top" command or press "q") > > > > > > > > > > ... > > > > > > > > > > Using busy-wait (with non-blocking sockets or non-blocking > I/O) > > : > > > > > -30 points > > > > > > > > > > So, this is a easy way to check if you are doing qualifies > > > > > as doing busy-wait (i.e., eating up all the CPU cycles on > > > > > nunki). If you are doing something like this, the fix is > > > > > simple! Right before you poll, sleep for 100 millisecond > > > > > before doing the poll. If that's too slow, sleep for 50 > > > > > millisecond. > > > > > -- > > > > > Bill Cheng // bill.cheng@usc.edu > http://merlot.usc.edu/william/usc/> Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 25 11:32:52 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3PIWqts017162 for ; Fri, 25 Apr 2008 11:32:52 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3PIAd4J001414 for ; Fri, 25 Apr 2008 11:10:39 -0700 Message-Id: <200804251810.m3PIAd4J001414@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Fri, 25 Apr 2008 11:10:39 -0700 From: Bill Cheng Someone wrote: > Even Aludra is taking the same time. I tried that option as well Looks like the same process, "sim-outorder", is taking up all the CPU cycles. By the way, does anyone know when the EE 557 project is due? -- Bill Cheng // bill.cheng@usc.edu On Fri, Apr 25, 2008 at 10:54 AM, Bill Cheng wrote: > Someone wrote: > > > Yesterday I tried working on an ITS SUN machine in SAL, my application > would > > start, and immediately before it gives me my command prompt, it will > > shutdown with no errors. I did not want to spend the time debugging the > > issue since I do not know if there is some limitation on the machine and > I > > would just be wasting my time, and also, if I am able to fix it for the > ITS > > machine, it might not work on nunki anymore, that is why I continued > working > > on nunki despite its issues. > > Hmm.. I do not know if this something with your code or it's > with the ITS SUN machines. I have used ITS SUN machines before > and had no problem and I have not heard anyone complained about > the ITS SUN machines! > > By the way, if compiling is taking too long, you can compile > on aludra! You can not run multiple processes on aludra. But > compiling should be fine. > -- > Bill Cheng // bill.cheng@usc.edu > > > > > On Fri, Apr 25, 2008 at 10:18 AM, Bill Cheng > wrote: > > > Someone wrote: > > > > > Nunki was practically not working yesterday. It was > taking > > > 10-15 minutes to compile the code. Not only there are cs551 projects > and > > > cs402 projects but also graphically heavy ee577 projects going on. > > > > Nunki is a shared machine and we have no control over it. > > As I mentioned during lectures, towards the end of the > > semesters, things can get crazy. Some classes will even be > > installing and running large systems (such as databases) on > > nunki and eat up all the system resources. Also, as you can > > see, some students in our class are not making things better. > > > > One solution is to work on an ITS workstation in SAL today! > > -- > > Bill Cheng // bill.cheng@usc.edu http://merlot.usc.edu/william/usc/> > > > > > > > > > > On Thu, Apr 24, 2008 at 11:18 AM, Bill Cheng > > > wrote: > > > Someone wrote: > > > > > > > Well, if the grader is grading the assignment after everyone has > > > > submitted their assignments (nunki is quiet), then sv_node will > > > > consume the top places on the CPU usage because it's the only > thing > > > > running. This doesn't mean it's busy waiting. Certificate > generation > > > > (I've noticed) is CPU-intensive, but a necessary step to complete > the > > > > project. I think such a guideline could be construed to unfairly > > > > reduce points on a project that doesn't busy wait. > > > > > > If your sv_node is giving up the processor, then it should > > > not show up near the top. If your job sleeps for 100ms, > > > after it wakes up, it polls (which take microseconds), and > > > goes right back to sleep. It should only use a tiny fraction > > > of the CPU! You can try the following yourself. Write a > > > small program that does the following: > > > > > > for (i=0; i < 500; i++) { > > > usleep(100000); > > > } > > > > > > run it and do "top" at the same time. This job should not > > > show up at all! > > > > > > You are correct that certificate generation can take some > > > time, that's why the grading guidelines says to wait 5 > > > seconds. I've just change the busy-wait to the following: > > > > > > in 4th window, wait 5 seconds, type: > > > top > > > (watch this for 10 seconds and press "q" or > > > to > > > exit "top") > > > (if sv_node is taking up the top spots in CPU > > percentages > > > and show high percentages, if so, this is considered > > > busy-wait and should take points off later, please > see > > > busy-wait in the negative points section) > > > > > > > One suggestion to detect this is to look for non-blocking IO. > Maybe > > > > look for the setsockopt that places it in the non-blocking mode. > > > > > > I think if you are eating up CPU cycles, you are doing some > > > sort of a busy-wait. If it's a strange bug, you should be > > > able to insert calls to usleep() to fix the problem! > > > > > > Finally, if the grader mistaken your code for busy-wait, you > > > can resolve it during regrading. So, if your code is not > > > staying in a tight loop waiting for some flags to change > > > state, there is nothing to worry about! > > > -- > > > Bill Cheng // bill.cheng@usc.edu > http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > > > > > > > On Apr 24, 2008, at 10:10 , Bill Cheng wrote: > > > > Someone wrote: > > > > > > > >> I'm trying to test on nunki now and have just seen the load > average > > > >> rise to and stay at for about 10 minutes 69.2. Since nunki has 4 > > > >> processors, this means that nunki is over 17 times over it's > optimal > > > >> load limit. It takes 20 minutes to build my project from scratch. > > > >> > > > >> This is making testing very difficult. Just letting you know > about > > > >> nunki's status ahead of time. > > > > > > > > Unfortunately, nunki is a shared machine and we cannot control > > > > other user. It may be best to work on an ITS workstation (I'm > > > > not sure if you can just ssh to one of them). *If* nunki goes > > > > down for over 2 hours between now and the project due time, > > > > please let me know and I will extend the deadline by one day. > > > > > > > > I also see that sv_node were sometimes eating up the CPU time > > > > on nunki. So, I'm increasing the "busy-wait" penalty to 30 > > > > points and added a test in (A.2.a) for it: > > > > > > > > in 4th window, wait 5 seconds, type: > > > > top > > > > (see if sv_node is taking up the top spots in CPU > > > > percentages, if so, this is considered busy-wait > > > > and should take points off later, please see busy-wait > > > > in the negative points section) > > > > (press to kill the "top" command or press "q") > > > > > > > > ... > > > > > > > > Using busy-wait (with non-blocking sockets or non-blocking I/O) > : > > > > -30 points > > > > > > > > So, this is a easy way to check if you are doing qualifies > > > > as doing busy-wait (i.e., eating up all the CPU cycles on > > > > nunki). If you are doing something like this, the fix is > > > > simple! Right before you poll, sleep for 100 millisecond > > > > before doing the poll. If that's too slow, sleep for 50 > > > > millisecond. > > > > -- > > > > Bill Cheng // bill.cheng@usc.edu http://merlot.usc.edu/william/usc/> Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 25 11:16:52 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3PIGqc1015719 for ; Fri, 25 Apr 2008 11:16:52 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3PHsc6w001129 for ; Fri, 25 Apr 2008 10:54:38 -0700 Message-Id: <200804251754.m3PHsc6w001129@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Fri, 25 Apr 2008 10:54:38 -0700 From: Bill Cheng Someone wrote: > Yesterday I tried working on an ITS SUN machine in SAL, my application would > start, and immediately before it gives me my command prompt, it will > shutdown with no errors. I did not want to spend the time debugging the > issue since I do not know if there is some limitation on the machine and I > would just be wasting my time, and also, if I am able to fix it for the ITS > machine, it might not work on nunki anymore, that is why I continued working > on nunki despite its issues. Hmm.. I do not know if this something with your code or it's with the ITS SUN machines. I have used ITS SUN machines before and had no problem and I have not heard anyone complained about the ITS SUN machines! By the way, if compiling is taking too long, you can compile on aludra! You can not run multiple processes on aludra. But compiling should be fine. -- Bill Cheng // bill.cheng@usc.edu On Fri, Apr 25, 2008 at 10:18 AM, Bill Cheng wrote: > Someone wrote: > > > Nunki was practically not working yesterday. It was taking > > 10-15 minutes to compile the code. Not only there are cs551 projects and > > cs402 projects but also graphically heavy ee577 projects going on. > > Nunki is a shared machine and we have no control over it. > As I mentioned during lectures, towards the end of the > semesters, things can get crazy. Some classes will even be > installing and running large systems (such as databases) on > nunki and eat up all the system resources. Also, as you can > see, some students in our class are not making things better. > > One solution is to work on an ITS workstation in SAL today! > -- > Bill Cheng // bill.cheng@usc.edu > > > > > On Thu, Apr 24, 2008 at 11:18 AM, Bill Cheng > wrote: > > Someone wrote: > > > > > Well, if the grader is grading the assignment after everyone has > > > submitted their assignments (nunki is quiet), then sv_node will > > > consume the top places on the CPU usage because it's the only thing > > > running. This doesn't mean it's busy waiting. Certificate generation > > > (I've noticed) is CPU-intensive, but a necessary step to complete the > > > project. I think such a guideline could be construed to unfairly > > > reduce points on a project that doesn't busy wait. > > > > If your sv_node is giving up the processor, then it should > > not show up near the top. If your job sleeps for 100ms, > > after it wakes up, it polls (which take microseconds), and > > goes right back to sleep. It should only use a tiny fraction > > of the CPU! You can try the following yourself. Write a > > small program that does the following: > > > > for (i=0; i < 500; i++) { > > usleep(100000); > > } > > > > run it and do "top" at the same time. This job should not > > show up at all! > > > > You are correct that certificate generation can take some > > time, that's why the grading guidelines says to wait 5 > > seconds. I've just change the busy-wait to the following: > > > > in 4th window, wait 5 seconds, type: > > top > > (watch this for 10 seconds and press "q" or > to > > exit "top") > > (if sv_node is taking up the top spots in CPU > percentages > > and show high percentages, if so, this is considered > > busy-wait and should take points off later, please see > > busy-wait in the negative points section) > > > > > One suggestion to detect this is to look for non-blocking IO. Maybe > > > look for the setsockopt that places it in the non-blocking mode. > > > > I think if you are eating up CPU cycles, you are doing some > > sort of a busy-wait. If it's a strange bug, you should be > > able to insert calls to usleep() to fix the problem! > > > > Finally, if the grader mistaken your code for busy-wait, you > > can resolve it during regrading. So, if your code is not > > staying in a tight loop waiting for some flags to change > > state, there is nothing to worry about! > > -- > > Bill Cheng // bill.cheng@usc.edu http://merlot.usc.edu/william/usc/> > > > > > > > > > > > > On Apr 24, 2008, at 10:10 , Bill Cheng wrote: > > > Someone wrote: > > > > > >> I'm trying to test on nunki now and have just seen the load average > > >> rise to and stay at for about 10 minutes 69.2. Since nunki has 4 > > >> processors, this means that nunki is over 17 times over it's optimal > > >> load limit. It takes 20 minutes to build my project from scratch. > > >> > > >> This is making testing very difficult. Just letting you know about > > >> nunki's status ahead of time. > > > > > > Unfortunately, nunki is a shared machine and we cannot control > > > other user. It may be best to work on an ITS workstation (I'm > > > not sure if you can just ssh to one of them). *If* nunki goes > > > down for over 2 hours between now and the project due time, > > > please let me know and I will extend the deadline by one day. > > > > > > I also see that sv_node were sometimes eating up the CPU time > > > on nunki. So, I'm increasing the "busy-wait" penalty to 30 > > > points and added a test in (A.2.a) for it: > > > > > > in 4th window, wait 5 seconds, type: > > > top > > > (see if sv_node is taking up the top spots in CPU > > > percentages, if so, this is considered busy-wait > > > and should take points off later, please see busy-wait > > > in the negative points section) > > > (press to kill the "top" command or press "q") > > > > > > ... > > > > > > Using busy-wait (with non-blocking sockets or non-blocking I/O) : > > > -30 points > > > > > > So, this is a easy way to check if you are doing qualifies > > > as doing busy-wait (i.e., eating up all the CPU cycles on > > > nunki). If you are doing something like this, the fix is > > > simple! Right before you poll, sleep for 100 millisecond > > > before doing the poll. If that's too slow, sleep for 50 > > > millisecond. > > > -- > > > Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 25 10:40:59 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3PHexgg012458 for ; Fri, 25 Apr 2008 10:40:59 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3PHIkJ2032692 for ; Fri, 25 Apr 2008 10:18:46 -0700 Message-Id: <200804251718.m3PHIkJ2032692@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Fri, 25 Apr 2008 10:18:46 -0700 From: Bill Cheng Someone wrote: > Nunki was practically not working yesterday. It was taking > 10-15 minutes to compile the code. Not only there are cs551 projects and > cs402 projects but also graphically heavy ee577 projects going on. Nunki is a shared machine and we have no control over it. As I mentioned during lectures, towards the end of the semesters, things can get crazy. Some classes will even be installing and running large systems (such as databases) on nunki and eat up all the system resources. Also, as you can see, some students in our class are not making things better. One solution is to work on an ITS workstation in SAL today! -- Bill Cheng // bill.cheng@usc.edu On Thu, Apr 24, 2008 at 11:18 AM, Bill Cheng wrote: > Someone wrote: > > > Well, if the grader is grading the assignment after everyone has > > submitted their assignments (nunki is quiet), then sv_node will > > consume the top places on the CPU usage because it's the only thing > > running. This doesn't mean it's busy waiting. Certificate generation > > (I've noticed) is CPU-intensive, but a necessary step to complete the > > project. I think such a guideline could be construed to unfairly > > reduce points on a project that doesn't busy wait. > > If your sv_node is giving up the processor, then it should > not show up near the top. If your job sleeps for 100ms, > after it wakes up, it polls (which take microseconds), and > goes right back to sleep. It should only use a tiny fraction > of the CPU! You can try the following yourself. Write a > small program that does the following: > > for (i=0; i < 500; i++) { > usleep(100000); > } > > run it and do "top" at the same time. This job should not > show up at all! > > You are correct that certificate generation can take some > time, that's why the grading guidelines says to wait 5 > seconds. I've just change the busy-wait to the following: > > in 4th window, wait 5 seconds, type: > top > (watch this for 10 seconds and press "q" or to > exit "top") > (if sv_node is taking up the top spots in CPU percentages > and show high percentages, if so, this is considered > busy-wait and should take points off later, please see > busy-wait in the negative points section) > > > One suggestion to detect this is to look for non-blocking IO. Maybe > > look for the setsockopt that places it in the non-blocking mode. > > I think if you are eating up CPU cycles, you are doing some > sort of a busy-wait. If it's a strange bug, you should be > able to insert calls to usleep() to fix the problem! > > Finally, if the grader mistaken your code for busy-wait, you > can resolve it during regrading. So, if your code is not > staying in a tight loop waiting for some flags to change > state, there is nothing to worry about! > -- > Bill Cheng // bill.cheng@usc.edu > > > > > > On Apr 24, 2008, at 10:10 , Bill Cheng wrote: > > Someone wrote: > > > >> I'm trying to test on nunki now and have just seen the load average > >> rise to and stay at for about 10 minutes 69.2. Since nunki has 4 > >> processors, this means that nunki is over 17 times over it's optimal > >> load limit. It takes 20 minutes to build my project from scratch. > >> > >> This is making testing very difficult. Just letting you know about > >> nunki's status ahead of time. > > > > Unfortunately, nunki is a shared machine and we cannot control > > other user. It may be best to work on an ITS workstation (I'm > > not sure if you can just ssh to one of them). *If* nunki goes > > down for over 2 hours between now and the project due time, > > please let me know and I will extend the deadline by one day. > > > > I also see that sv_node were sometimes eating up the CPU time > > on nunki. So, I'm increasing the "busy-wait" penalty to 30 > > points and added a test in (A.2.a) for it: > > > > in 4th window, wait 5 seconds, type: > > top > > (see if sv_node is taking up the top spots in CPU > > percentages, if so, this is considered busy-wait > > and should take points off later, please see busy-wait > > in the negative points section) > > (press to kill the "top" command or press "q") > > > > ... > > > > Using busy-wait (with non-blocking sockets or non-blocking I/O) : > > -30 points > > > > So, this is a easy way to check if you are doing qualifies > > as doing busy-wait (i.e., eating up all the CPU cycles on > > nunki). If you are doing something like this, the fix is > > simple! Right before you poll, sleep for 100 millisecond > > before doing the poll. If that's too slow, sleep for 50 > > millisecond. > > -- > > Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 24 12:20:11 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=AWL,BAYES_05 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3OJKBaH020463 for ; Thu, 24 Apr 2008 12:20:11 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3OIw5wX009146 for ; Thu, 24 Apr 2008 11:58:05 -0700 Message-Id: <200804241858.m3OIw5wX009146@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Changes in the grading guidelines Date: Thu, 24 Apr 2008 11:58:05 -0700 From: Bill Cheng Someone wrote: > In the test that you have added for get with external file, > you are testing for blondie.mp3 instead of blondie_2.mp3. The > test should be > > in the 4th window, type at the UNIX prompt: > "ls -l blondie_2.mp3" > "openssl sha1 blondie_2.mp3" > "rm blondie_2.mp3". > > Currently it is > > in the 4th window, type at the UNIX prompt: > "ls -l blondie1.mp3" > "openssl sha1 blondie1.mp3" > "rm blondie_2.mp3" > Oops! Fixed! Thanks! -- Bill Cheng // bill.cheng@usc.edu ----- Original Message ----- From: Bill Cheng Date: Thursday, April 24, 2008 11:27 am Subject: Re: Changes in the grading guidelines To: cs551@merlot.usc.edu > Someone wrote: > > > In the test case (B) (1) (a) search filename=earc.jpg test case > > its written that there should be 5 responses but there can be 4 > > responses also as the node *01 > > Fixed! Thanks for letting me know. > > > Also there is no test for get with an external file name. > > I've replaced a "get 3" in (A.2.a) to "get 3 blondie_2.mp3". > -- > Bill Cheng // bill.cheng@usc.edu > Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 24 12:17:59 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3OJHwTN020044 for ; Thu, 24 Apr 2008 12:17:58 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3OItrOH009073 for ; Thu, 24 Apr 2008 11:55:53 -0700 Message-Id: <200804241855.m3OItrOH009073@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt Final2 Date: Thu, 24 Apr 2008 11:55:53 -0700 From: Bill Cheng Someone wrote: > For the files::: > sha1_index,fileid_index,kwrd_index... > is there a syntax that we have to folllow,or I can store data into these > files in any order I want. Anyway you want. But since these are *index* files, data items should be sorted in some way. You should document this in your README file. > And will you be checking these files during grading,and do they have marks > alloted to them. Please see the grading guidelines (in the minus points section). > Also what goes into kwrd_index,the bit vectors or the keywords . You don't really need the keywords, although it's nice to have them since you won't need to go the metadata files to retrieve them. So, the minimum entry for a kwrd_index entry should be: bit-vector -> list of file numbers with the same bit-vector -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 24 11:48:40 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3OImevQ017556 for ; Thu, 24 Apr 2008 11:48:40 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3OIQZ5O008427 for ; Thu, 24 Apr 2008 11:26:35 -0700 Message-Id: <200804241826.m3OIQZ5O008427@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Changes in the grading guidelines Date: Thu, 24 Apr 2008 11:26:35 -0700 From: Bill Cheng Someone wrote: > In the test case (B) (1) (a) search filename=earc.jpg test case > its written that there should be 5 responses but there can be 4 > responses also as the node *01 Fixed! Thanks for letting me know. > Also there is no test for get with an external file name. I've replaced a "get 3" in (A.2.a) to "get 3 blondie_2.mp3". -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 24 11:40:56 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3OIeuI8016762 for ; Thu, 24 Apr 2008 11:40:56 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3OIIpRg007984 for ; Thu, 24 Apr 2008 11:18:51 -0700 Message-Id: <200804241818.m3OIIpRg007984@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Thu, 24 Apr 2008 11:18:51 -0700 From: Bill Cheng Someone wrote: > Well, if the grader is grading the assignment after everyone has > submitted their assignments (nunki is quiet), then sv_node will > consume the top places on the CPU usage because it's the only thing > running. This doesn't mean it's busy waiting. Certificate generation > (I've noticed) is CPU-intensive, but a necessary step to complete the > project. I think such a guideline could be construed to unfairly > reduce points on a project that doesn't busy wait. If your sv_node is giving up the processor, then it should not show up near the top. If your job sleeps for 100ms, after it wakes up, it polls (which take microseconds), and goes right back to sleep. It should only use a tiny fraction of the CPU! You can try the following yourself. Write a small program that does the following: for (i=0; i < 500; i++) { usleep(100000); } run it and do "top" at the same time. This job should not show up at all! You are correct that certificate generation can take some time, that's why the grading guidelines says to wait 5 seconds. I've just change the busy-wait to the following: in 4th window, wait 5 seconds, type: top (watch this for 10 seconds and press "q" or to exit "top") (if sv_node is taking up the top spots in CPU percentages and show high percentages, if so, this is considered busy-wait and should take points off later, please see busy-wait in the negative points section) > One suggestion to detect this is to look for non-blocking IO. Maybe > look for the setsockopt that places it in the non-blocking mode. I think if you are eating up CPU cycles, you are doing some sort of a busy-wait. If it's a strange bug, you should be able to insert calls to usleep() to fix the problem! Finally, if the grader mistaken your code for busy-wait, you can resolve it during regrading. So, if your code is not staying in a tight loop waiting for some flags to change state, there is nothing to worry about! -- Bill Cheng // bill.cheng@usc.edu On Apr 24, 2008, at 10:10 , Bill Cheng wrote: > Someone wrote: > >> I'm trying to test on nunki now and have just seen the load average >> rise to and stay at for about 10 minutes 69.2. Since nunki has 4 >> processors, this means that nunki is over 17 times over it's optimal >> load limit. It takes 20 minutes to build my project from scratch. >> >> This is making testing very difficult. Just letting you know about >> nunki's status ahead of time. > > Unfortunately, nunki is a shared machine and we cannot control > other user. It may be best to work on an ITS workstation (I'm > not sure if you can just ssh to one of them). *If* nunki goes > down for over 2 hours between now and the project due time, > please let me know and I will extend the deadline by one day. > > I also see that sv_node were sometimes eating up the CPU time > on nunki. So, I'm increasing the "busy-wait" penalty to 30 > points and added a test in (A.2.a) for it: > > in 4th window, wait 5 seconds, type: > top > (see if sv_node is taking up the top spots in CPU > percentages, if so, this is considered busy-wait > and should take points off later, please see busy-wait > in the negative points section) > (press to kill the "top" command or press "q") > > ... > > Using busy-wait (with non-blocking sockets or non-blocking I/O) : > -30 points > > So, this is a easy way to check if you are doing qualifies > as doing busy-wait (i.e., eating up all the CPU cycles on > nunki). If you are doing something like this, the fix is > simple! Right before you poll, sleep for 100 millisecond > before doing the poll. If that's too slow, sleep for 50 > millisecond. > -- > Bill Cheng // bill.cheng@usc.edu ; Thu, 24 Apr 2008 10:32:39 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3OHAYpr005857 for ; Thu, 24 Apr 2008 10:10:34 -0700 Message-Id: <200804241710.m3OHAYpr005857@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Thu, 24 Apr 2008 10:10:34 -0700 From: Bill Cheng Someone wrote: > I'm trying to test on nunki now and have just seen the load average > rise to and stay at for about 10 minutes 69.2. Since nunki has 4 > processors, this means that nunki is over 17 times over it's optimal > load limit. It takes 20 minutes to build my project from scratch. > > This is making testing very difficult. Just letting you know about > nunki's status ahead of time. Unfortunately, nunki is a shared machine and we cannot control other user. It may be best to work on an ITS workstation (I'm not sure if you can just ssh to one of them). *If* nunki goes down for over 2 hours between now and the project due time, please let me know and I will extend the deadline by one day. I also see that sv_node were sometimes eating up the CPU time on nunki. So, I'm increasing the "busy-wait" penalty to 30 points and added a test in (A.2.a) for it: in 4th window, wait 5 seconds, type: top (see if sv_node is taking up the top spots in CPU percentages, if so, this is considered busy-wait and should take points off later, please see busy-wait in the negative points section) (press to kill the "top" command or press "q") ... Using busy-wait (with non-blocking sockets or non-blocking I/O) : -30 points So, this is a easy way to check if you are doing qualifies as doing busy-wait (i.e., eating up all the CPU cycles on nunki). If you are doing something like this, the fix is simple! Right before you poll, sleep for 100 millisecond before doing the poll. If that's too slow, sleep for 50 millisecond. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 24 09:49:42 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3OGngfJ006523 for ; Thu, 24 Apr 2008 09:49:42 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3OGRbkY004298 for ; Thu, 24 Apr 2008 09:27:37 -0700 Message-Id: <200804241627.m3OGRbkY004298@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 2 Date: Thu, 24 Apr 2008 09:27:37 -0700 From: Bill Cheng Someone wrote: > In the test case (A)(3) when the store command is issued for the > chess.jpg file the keywords are info="photo jpeg". But when you > have asked to verify the meta data of the file, the keywords are > shown as 'categories audio mp3 artist Blondie'. Oops! Fixed! Thanks for catching it! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 22:45:22 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3O5jM6s011099 for ; Wed, 23 Apr 2008 22:45:22 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3O5NLoL025748 for ; Wed, 23 Apr 2008 22:23:21 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m3O5NLqE025747 for cs551@merlot; Wed, 23 Apr 2008 22:23:21 -0700 Date: Wed, 23 Apr 2008 22:23:21 -0700 From: Bill Cheng Message-Id: <200804240523.m3O5NLqE025747@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: part (2) partners... Hi everyone, If you and your partner has sent me a commitment forms for part (1) of the final project *only*, and you still want to be partners, please make sure you and your partner both send me a commitment for part (2) by the part (2) submission deadline. If you decide to not to be partners with your partner of part (1), it's perfectly okay to start with the same part (1) code. But you should let me know by e-mail that this is going on. Thanks! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 22:39:43 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3O5dhMX010568 for ; Wed, 23 Apr 2008 22:39:43 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3O5HgY7025626 for ; Wed, 23 Apr 2008 22:17:42 -0700 Message-Id: <200804240517.m3O5HgY7025626@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Problem with openssl and bash Date: Wed, 23 Apr 2008 22:17:42 -0700 From: Bill Cheng Someone wrote: > I am getting random behavior when I try to use popen to > execute the commands to > 1. Sign the filespec > 2. Verify the signed file > > A few times it would run perfectly but most of the time > it gives me an error: > > bash: line 1: 1: Bad file number. I assume this means that it's complaining about the "2>&1" since the 2 and 1 there are file numbers. May be you have some other numeric values in your command? > If i run the same command multiple times (with different "-out file"), > sometimes it runs for > the first command but fails after that. If the same request is > forwarded to multiple nodes, > it might run for some of the nodes and not for others.. I'm not sure why you would "forward" the command to other nodes. Each node should execute this command on its own and use the appropriate directory name (somewhere in its HomeDir). > I have checked the code to make sure all files which are required are > present (example: i > read the filespec i just created before running the command to make > sure it exists.) and > also that those files arent being accessed. > > The code I am using is what has been mentioned in the spec. > > Could you please suggest what I should do? I would run the command in a separate small program and make sure it works every time. If it does work, then you just need to find out the difference between your code and that small program. By the way, are you using fullpath names for the files? The code segment in the spec says something like: "... -signer cert.pem ..." you should replace "cert.pem" with the fullpath name to the actual certificate file. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 22:18:05 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3O5I4DL008748 for ; Wed, 23 Apr 2008 22:18:05 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3O4u4fa025236 for ; Wed, 23 Apr 2008 21:56:04 -0700 Message-Id: <200804240456.m3O4u4fa025236@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551:final project 2 grading guideline Date: Wed, 23 Apr 2008 21:56:04 -0700 From: Bill Cheng Someone wrote: > In the message attached below, you have mentioned > > > If a node already has a file (i.e., same FileName, SHA1, and > > Nonce), it should not save a copy of the file. > > But in test case: A (2) (a) where we do: > > ----------------- > in 3rd window, type: > (+1 points) > "get 2" > ----------------- > > Shouldn't the file recd from "get 2" be discarded as it is a duplicate > of the file saved when we did store since (because of the store) the > Nonce, SHA1 and Filename will be same for that file stored on all > nodes. > > Or are we supposed to copy the file to the current working directory > but not to the node's own filesystem if its a duplicate file? Exactly! -- Bill Cheng // bill.cheng@usc.edu On Mon, Apr 21, 2008 at 8:23 AM, Bill Cheng wrote: > Someone wrote: > > > In the grading guideline section (B)(1)(a), after we did "get > > 1-5" on node *08, the guideline states that > > > > [type "status files 5 05.out", look at "05.out", > > all nodes should have "earc.jpg" and each > > node should only have one copy of it.] > > > > I don't understand why each node should "only have one copy" of > > it. My understanding is that whenever node *08 > > issues a GET request message, the immediate nodes along the > > return path (from the target node to node *08) would > > probabilistically cache the GET response file (in the above test > > case, immediate node will always cache the response file since > > CacheProb=1 and CacheSize=5) even if the same copy of the file( > > same filename, sha1, and nonce) has already been cached. > > Therefore what I got is node *08 has 4 permanent copies of the > > "earc.jpg", and some other nodes has more than 1 cache copy of > > the "earc.jpg", especially the neighbors of node *08. > > > > Please let me know whether my assumption is correct (each node is > > allowed to have more than one cache copy of any particular file > > which implies that all cache copies of that file would have the > > same filename, sha1, and nonce). > > If a node already has a file (i.e., same FileName, SHA1, and > Nonce), it should not save a copy of the file. In my message > with timestamp "Sun 13 Apr 16:59:21", I mentioned this for > the node that initiated the GET. But the principle that "a file > with the same FileName, SHA1, and Nonce should appear on a node > at most once" should apply to all nodes. > -- > Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 16:25:36 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=AWL,BAYES_20 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3NNPaWt010122 for ; Wed, 23 Apr 2008 16:25:36 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3NN3cJ1020109 for ; Wed, 23 Apr 2008 16:03:38 -0700 Message-Id: <200804232303.m3NN3cJ1020109@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP2: missing ini files for the new LRU tests Date: Wed, 23 Apr 2008 16:03:38 -0700 From: Bill Cheng Someone wrote: The ini files are still not available. Am I using some old version of guidelines.. I have refreshed the page many a times but the ini files are not there at ~/grading/guidelines/final2 You either need to rerun "~csci551b/bin/setupf2 $HOME" or you can copy these ini files from ~csci551b/public/final2 manually: cp ~csci551b/public/final2/b3-*.ini ~/grading/guidelines/final2 I thought you were asking about the links on the following web page: http://merlot.usc.edu/cs551-s08/projects/testcases/p2.html -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- From: Bill Cheng [mailto:william@bourbon.usc.edu] Sent: Wednesday, April 23, 2008 3:21 PM To: cs551@merlot.usc.edu Subject: Re: CS551: FP2: missing ini files for the new LRU tests Someone wrote: > In the newly introduced LRU test cases (+6 points), the ini files are > missing for node 01 and node 02. Please make them available so that we can > have complete testing of our LUR implementation. I've updated the web page. I've also discovered that nodes 01 and 02 are not beacon nodes! So, I've also fixed that. Now nodes 01, 02, and 03 are all beacon nodes and I don't use node 00 in (A.3). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 15:47:25 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3NMlPGF006638 for ; Wed, 23 Apr 2008 15:47:25 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3NMPQ8Z019069 for ; Wed, 23 Apr 2008 15:25:26 -0700 Message-Id: <200804232225.m3NMPQ8Z019069@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Unusual timing issue with part (2) grading guidelines... Date: Wed, 23 Apr 2008 15:25:26 -0700 From: Bill Cheng Someone wrote: > What can be a reason for this timing issue? If TTL is given as 2, a file > originated at node 02 must reach both the beacon nodes as both are only two > hop away. The given topology is the following: topology: /------ n08 | | v v n00* <--> n01* ^^ ^ || | |\------ n04 <----\ | ^ | | | | \------- n03 <-- n02 If you flood a message with TTL=2 at node 02, you can expect that this message will reach node 01. But, it is possible that right after node 02 sent the message to node 03, it got suspended by the OS before it has a chance to send the same message to node 04. Now node 03 forwards the message to nodes 00 and 04. When node 02 wakes up and send the message to node 04, node 04 will see that the message is duplicate and drop it! So, the message never reaches node 01. -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- From: Bill Cheng [mailto:william@bourbon.usc.edu] Sent: Wednesday, April 23, 2008 10:39 AM To: cs551@merlot.usc.edu Subject: Unusual timing issue with part (2) grading guidelines... Hi everyone, Someone pointed out that there are a couple of places in the tests in (B.1.a) where an alternative solutions are possible. This happens when TTL=2 for a store command in the network topology for section (B). For "store earc.jpg 2 ..." initiated at node *02, it is possible that the STORE message may not even reach node *01. Similarly, for "store usctommy.gif 2 ..." initiated at node *08, it is possible that the STORE message may not even reach node *03. I have updated the grading guidelines stating these possibilities. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 15:42:33 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3NMgXaY006276 for ; Wed, 23 Apr 2008 15:42:33 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3NMKYGM018886 for ; Wed, 23 Apr 2008 15:20:34 -0700 Message-Id: <200804232220.m3NMKYGM018886@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP2: missing ini files for the new LRU tests Date: Wed, 23 Apr 2008 15:20:34 -0700 From: Bill Cheng Someone wrote: > In the newly introduced LRU test cases (+6 points), the ini files are > missing for node 01 and node 02. Please make them available so that we can > have complete testing of our LUR implementation. I've updated the web page. I've also discovered that nodes 01 and 02 are not beacon nodes! So, I've also fixed that. Now nodes 01, 02, and 03 are all beacon nodes and I don't use node 00 in (A.3). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 14:42:27 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3NLgRcP000681 for ; Wed, 23 Apr 2008 14:42:27 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3NLKTqV017180 for ; Wed, 23 Apr 2008 14:20:29 -0700 Message-Id: <200804232120.m3NLKTqV017180@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: LRU tests are missing in the grading guidelines! Date: Wed, 23 Apr 2008 14:20:29 -0700 From: Bill Cheng I've found a simple fix. But this would require you to add two more files "b3-n01.ini" and "b3-n02.ini". I've moved (A.2.d) to (A.3) and use a 10KB CacheSize. Sorry about the inconvenience. If there are still bugs, please let me know. Thanks! -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- Date: Wed, 23 Apr 2008 14:07:38 -0700 From: Bill Cheng To: cs551@merlot.usc.edu Subject: Re: LRU tests are missing in the grading guidelines! Someone wrote: > you havent changed the ini file. the cache size is 600 in the ini > file which is equal to 600k You are right! CacheSize is 10KB for (A.1) and 600KB for (A.2). I need to find a way to change this! Sorry about the bug. I'll send an update soon. -- Bill Cheng // bill.cheng@usc.edu ----- Original Message ----- From: Bill Cheng Date: Thursday, April 24, 2008 12:53 am Subject: LRU tests are missing in the grading guidelines! To: cs551@merlot.usc.edu > Hi everyone, > > Someone mentioned this week that there were no tests for LRU > in the grading guidelines. I've just added (A.2.d) for this. > It's worth 6 points. I've removed 1 point from documentation. > So, the total number of points for part (2) is 90. (By the > way, the total number of points doesn't really mean anything > since everything will be normalized to 100 points before they > are weighted and summed.) > -- > Bill Cheng // bill.cheng@usc.edu > Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 14:29:36 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3NLTadp031716 for ; Wed, 23 Apr 2008 14:29:36 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3NL7cq8016828 for ; Wed, 23 Apr 2008 14:07:38 -0700 Message-Id: <200804232107.m3NL7cq8016828@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: LRU tests are missing in the grading guidelines! Date: Wed, 23 Apr 2008 14:07:38 -0700 From: Bill Cheng Someone wrote: > you havent changed the ini file. the cache size is 600 in the ini > file which is equal to 600k You are right! CacheSize is 10KB for (A.1) and 600KB for (A.2). I need to find a way to change this! Sorry about the bug. I'll send an update soon. -- Bill Cheng // bill.cheng@usc.edu ----- Original Message ----- From: Bill Cheng Date: Thursday, April 24, 2008 12:53 am Subject: LRU tests are missing in the grading guidelines! To: cs551@merlot.usc.edu > Hi everyone, > > Someone mentioned this week that there were no tests for LRU > in the grading guidelines. I've just added (A.2.d) for this. > It's worth 6 points. I've removed 1 point from documentation. > So, the total number of points for part (2) is 90. (By the > way, the total number of points doesn't really mean anything > since everything will be normalized to 100 points before they > are weighted and summed.) > -- > Bill Cheng // bill.cheng@usc.edu > Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 12:45:37 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3NJjbn6022488 for ; Wed, 23 Apr 2008 12:45:37 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3NJNe8H014268 for ; Wed, 23 Apr 2008 12:23:40 -0700 Message-Id: <200804231923.m3NJNe8H014268@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: LRU tests are missing in the grading guidelines! Date: Wed, 23 Apr 2008 12:23:40 -0700 From: Bill Cheng By the way, if you have implemented LRU, please let me know if you see any bugs in (A.2.d). Thanks! -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- Date: Wed, 23 Apr 2008 12:22:02 -0700 From: Bill Cheng To: cs551@merlot.usc.edu Subject: LRU tests are missing in the grading guidelines! Hi everyone, Someone mentioned this week that there were no tests for LRU in the grading guidelines. I've just added (A.2.d) for this. It's worth 6 points. I've removed 1 point from documentation. So, the total number of points for part (2) is 90. (By the way, the total number of points doesn't really mean anything since everything will be normalized to 100 points before they are weighted and summed.) -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 12:44:00 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3NJi0A8022294 for ; Wed, 23 Apr 2008 12:44:00 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3NJM2FI014215 for ; Wed, 23 Apr 2008 12:22:02 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m3NJM2gF014214 for cs551@merlot; Wed, 23 Apr 2008 12:22:02 -0700 Date: Wed, 23 Apr 2008 12:22:02 -0700 From: Bill Cheng Message-Id: <200804231922.m3NJM2gF014214@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: LRU tests are missing in the grading guidelines! Hi everyone, Someone mentioned this week that there were no tests for LRU in the grading guidelines. I've just added (A.2.d) for this. It's worth 6 points. I've removed 1 point from documentation. So, the total number of points for part (2) is 90. (By the way, the total number of points doesn't really mean anything since everything will be normalized to 100 points before they are weighted and summed.) -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 11:01:22 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3NI1MZJ012889 for ; Wed, 23 Apr 2008 11:01:22 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3NHdPaB010449 for ; Wed, 23 Apr 2008 10:39:25 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m3NHdP9w010448 for cs551@merlot; Wed, 23 Apr 2008 10:39:25 -0700 Date: Wed, 23 Apr 2008 10:39:25 -0700 From: Bill Cheng Message-Id: <200804231739.m3NHdP9w010448@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Unusual timing issue with part (2) grading guidelines... Hi everyone, Someone pointed out that there are a couple of places in the tests in (B.1.a) where an alternative solutions are possible. This happens when TTL=2 for a store command in the network topology for section (B). For "store earc.jpg 2 ..." initiated at node *02, it is possible that the STORE message may not even reach node *01. Similarly, for "store usctommy.gif 2 ..." initiated at node *08, it is possible that the STORE message may not even reach node *03. I have updated the grading guidelines stating these possibilities. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 23 10:53:13 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3NHrDPJ012175 for ; Wed, 23 Apr 2008 10:53:13 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3NHVG7S010251 for ; Wed, 23 Apr 2008 10:31:16 -0700 Message-Id: <200804231731.m3NHVG7S010251@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 2 Date: Wed, 23 Apr 2008 10:31:16 -0700 From: Bill Cheng Someone wrote: > In the minus points section of the grading guideline under the > heading 'Failing kills', shouldn't the file that is copied be > taken from the directory 'final2'? In the guideline it is taken > from final1. Fixed! Thanks for catching this. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Apr 21 22:41:26 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3M5fQ98009934 for ; Mon, 21 Apr 2008 22:41:26 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3M5JgGP032651 for ; Mon, 21 Apr 2008 22:19:42 -0700 Message-Id: <200804220519.m3M5JgGP032651@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: doubt in search Date: Mon, 21 Apr 2008 22:19:42 -0700 From: Bill Cheng Someone wrote: > If i get a search command on my node, then do i need to > search only in my cache or in my permanent too ?? Please remember that other than the fact that a cached file is subject to LRU, there is really no difference between cached or permanent files. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Apr 21 08:45:21 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3LFjKqh032191 for ; Mon, 21 Apr 2008 08:45:20 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3LFNfxT014107 for ; Mon, 21 Apr 2008 08:23:41 -0700 Message-Id: <200804211523.m3LFNfxT014107@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551:final project 2 grading guideline Date: Mon, 21 Apr 2008 08:23:41 -0700 From: Bill Cheng Someone wrote: > In the grading guideline section (B)(1)(a), after we did "get > 1-5" on node *08, the guideline states that > > [type "status files 5 05.out", look at "05.out", > all nodes should have "earc.jpg" and each > node should only have one copy of it.] > > I don't understand why each node should "only have one copy" of > it. My understanding is that whenever node *08 > issues a GET request message, the immediate nodes along the > return path (from the target node to node *08) would > probabilistically cache the GET response file (in the above test > case, immediate node will always cache the response file since > CacheProb=1 and CacheSize=5) even if the same copy of the file( > same filename, sha1, and nonce) has already been cached. > Therefore what I got is node *08 has 4 permanent copies of the > "earc.jpg", and some other nodes has more than 1 cache copy of > the "earc.jpg", especially the neighbors of node *08. > > Please let me know whether my assumption is correct (each node is > allowed to have more than one cache copy of any particular file > which implies that all cache copies of that file would have the > same filename, sha1, and nonce). If a node already has a file (i.e., same FileName, SHA1, and Nonce), it should not save a copy of the file. In my message with timestamp "Sun 13 Apr 16:59:21", I mentioned this for the node that initiated the GET. But the principle that "a file with the same FileName, SHA1, and Nonce should appear on a node at most once" should apply to all nodes. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 20 20:54:28 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=AWL,BAYES_20 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3L3sSYk032635 for ; Sun, 20 Apr 2008 20:54:28 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3L3WqQC001909 for ; Sun, 20 Apr 2008 20:32:52 -0700 Message-Id: <200804210332.m3L3WqQC001909@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551:Regarding final project. Date: Sun, 20 Apr 2008 20:32:52 -0700 From: Bill Cheng Someone wrote: > My query is regarding Delete message: > > The scenario is : > > 3 nodes - A , B , C... all are up. > A file X is stored by executing store command on node A and it is > cached at Node B only . > > C , then , does a get request for file X and it is stored in its > permanent storage and current working directory. > > Consider the case when node B and C nodes are down and we execute > a delete command at A. > After successful delete at A , we would not be able to execute > the delete for this file again . I'm not sure why you would say this. You can issue the same delete command as many times are you want. > Now , when Node B and C come up , they would never receive a > delete request for the file X. > So as a result , it might cache out in B but in C it would stay > in the permanent storage forever. Yes. But you can issue the same delete command again at the same node as before. > Is this the right interpretation of the delete message flow or am > i missing some point that would avoid the above situation? What you have described is not an error. I've mentioned this in class before and this is the nature of a peer-to-peer system (since a node can be down). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 20 20:45:48 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3L3jmhH031760 for ; Sun, 20 Apr 2008 20:45:48 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3L3OCOF001815 for ; Sun, 20 Apr 2008 20:24:12 -0700 Message-Id: <200804210324.m3L3OCOF001815@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP2: make clean query Date: Sun, 20 Apr 2008 20:24:12 -0700 From: Bill Cheng Someone wrote: > Should we delete mp3, wav, gif, jpg, out files also using make clean? The Makefile page says: When the following command is invoked at the UNIX prompt: make clean all binary files created during compilation (.o files and executable files) must be removed. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 20 14:10:29 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3KLATY3029024 for ; Sun, 20 Apr 2008 14:10:29 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3KKmttg027754 for ; Sun, 20 Apr 2008 13:48:55 -0700 Message-Id: <200804202048.m3KKmttg027754@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS-551: FP2: some queries Date: Sun, 20 Apr 2008 13:48:55 -0700 From: Bill Cheng Someone wrote: > I have some doubts and requests for the final part 2: > > 1. In the grading guidelines part (B - mix of nodes), the AutoShutdown time > for all the nodes seems too small to finish all the test cases. My nodes > always shut down before I am able to finish the tests. Will it be possible > for you to increase the time by 50-70% for all the six nodes so that I can > finish the tests completely. The grading guidelines is meant to be used by the grader during grading. SO, if there is not enough time because it takes you too long to type things in different windows, then it's not an issue because the grader will be fast. If your nodes are running really slow, then you should find out why and may be fix it. > 2. In the log file, is it OK to keep some white space between words to align > the entries properly. I mean is it ok to print 'length' and 'size' as %3d > and %6d respectively? Or there must be only one space between two words? Having extra white space is fine as long as it makes the log file *more* readable. > 3. When a node receives a 'flooding' message like GTRQ or STRQ, should it > make the flooded entries as 'f' or 's' in the logfile. I mean does a node > 'sends' or 'forwards' a flooding message? "s" means originating a message and "f" means forwarding a message. > 4. When a originator node floods a message, should it make the entries as > 'forward' or 'send' in its log file. Since it's not forwading a message, it should be "s". -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 20 09:41:37 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3KGfaLv005000 for ; Sun, 20 Apr 2008 09:41:36 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3KGK4go022957 for ; Sun, 20 Apr 2008 09:20:04 -0700 Message-Id: <200804201620.m3KGK4go022957@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 FP2 : Problem with sending .wav/.jpg file Date: Sun, 20 Apr 2008 09:20:04 -0700 From: Bill Cheng Someone wrote: > We are facing a problem when sending a .wav file or a .jpg file from one > beacon node to another beacon node. The file is COMPLETELY sent from the > sender side, but the file seems to be received in chunks at the receiver > side. Even though we are sending file byte-by-byte, and receiving > byte-by-byte, the receiver is receiving the bytes very slowly. Moreover, > after the entire file has been received the number of bytes received is same > as number of bytes sent. But now the SHA1 values do not match. Again, If I > try sending a text file like .cpp or .c files then we correctly receive > these files at the receiver and their SHA1 values do match. Please advise. I have no explanation regarding why it's slow when you read a binary file but it's fast when you read a text file. My guess is that it's a bug and you need to find it. The reason you get not receiving binary files correct is probably because you used functions for reading and writing ASCII data (such as fgets() and fprintf()). I would have thought that you've dealt with this bug before in warmup project #1. To debug this should be easy. The first 16 bytes of a JPEG file usually have some zeroes in it. Just debug the first 16 bytes you send. In the sender, right before you call send() or write(), examine the buffer in the debugger. Do the following in the debugger: printf "0x%02x\n", buf[0] printf "0x%02x\n", buf[1] printf "0x%02x\n", buf[2] printf "0x%02x\n", buf[3] printf "0x%02x\n", buf[4] ... printf "0x%02x\n", buf[15] If they look okay, do something similar at the receiver. Right after your call to read() or recv(), examine the buffer in the debugger in a similar way. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 18 23:10:41 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3J6Aeuf012389 for ; Fri, 18 Apr 2008 23:10:40 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3J5nKBN031452 for ; Fri, 18 Apr 2008 22:49:20 -0700 Message-Id: <200804190549.m3J5nKBN031452@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: nunki problems Date: Fri, 18 Apr 2008 22:49:20 -0700 From: Bill Cheng Someone wrote: > nunki has 98% cpu usage since yesterday. I cannot work at all and I > inspect that there's few user who run those big program which take much > of the cpu. > Is there a way to suspend those users? Can you help contact admin? There is no way to suspend these users, as far as I know. The system will kill a job when a job has used up it's CPU quota. Please note that, according to the grading guidelines, you will lose 20 points if your program does busy-wait and eats up all the CPU cycles. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 18 15:43:37 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3IMhb1e003789 for ; Fri, 18 Apr 2008 15:43:37 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3IMMKos026499 for ; Fri, 18 Apr 2008 15:22:20 -0700 Message-Id: <200804182222.m3IMMKos026499@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt with delete message Date: Fri, 18 Apr 2008 15:22:20 -0700 From: Bill Cheng Someone wrote: > I have a doubt regarding delete message. > The specs say > > "To delete a file named foo, a user should enter: > delete FileName=foo SHA1=6b6c... Nonce=fe18..." > > Let us suppose that the user enters a correct filename (say foo) > but SHA1 that is not the same as the of foo. In this scenarios > what should we do. If we delete the file using filenameindex, > this could leave the system inconsistent. Even if you use filename index, you should verify Nonce and SHA1 (even before you verify the digitial signature). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 18 11:00:26 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3II0QAQ011267 for ; Fri, 18 Apr 2008 11:00:26 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3IHdAmG019999 for ; Fri, 18 Apr 2008 10:39:10 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m3IHdAE2019998 for cs551@merlot; Fri, 18 Apr 2008 10:39:10 -0700 Date: Fri, 18 Apr 2008 10:39:10 -0700 From: Bill Cheng Message-Id: <200804181739.m3IHdAE2019998@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Final exam location will be the same as midterm! Hi everyone, Just want to let you know that the final exam for CS 551 will be in MHP 106 (same room as midterm) from 2pm to 4pm on Wednesday, 5/14/2008. I have updated the class web page with the room information. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 18 00:02:56 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3I72uVe016693 for ; Fri, 18 Apr 2008 00:02:56 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3I6fiPj008311 for ; Thu, 17 Apr 2008 23:41:44 -0700 Message-Id: <200804180641.m3I6fiPj008311@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: doubt in FP2 Date: Thu, 17 Apr 2008 23:41:44 -0700 From: Bill Cheng Someone wrote: > While search when a node has to search in its own file section, Should it > search for only the cache files or all the files(cache + permanent) > > What should be done when the search message is forwarded to other nodes? Please see the bottom of my message with timestamp "Wed 16 Apr 23:08". -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 17 10:50:33 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3HHoXVW011480 for ; Thu, 17 Apr 2008 10:50:33 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3HHTPkq018140 for ; Thu, 17 Apr 2008 10:29:25 -0700 Message-Id: <200804171729.m3HHTPkq018140@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP2: question Date: Thu, 17 Apr 2008 10:29:25 -0700 From: Bill Cheng Someone wrote: > I know this one is a silly question, but I wanted to be cent percent sure > before submission: > > Q: If a user enters "search sha1hash=34abd3de", should we flood the meesage > and wait for 1.2*MsgLifeTime (or till the user presses ^C) OR we can give a > message like 'ERROR: sha1 should be a string of 40 char' and give the > command prompt back to the user instantly. Either way would be correct. But giving an error and not flood is better! > Also, is it appropriate to display some meaningful messages after certain > commands, e.g. 'get: file placed in current directory successfully' after a > get command. Absolutely! And this is not a debugging message. It's good to keep the user informed about what is happening and what has happened. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 17 00:15:25 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3H7FPfn019822 for ; Thu, 17 Apr 2008 00:15:25 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3H6sLkm006888 for ; Wed, 16 Apr 2008 23:54:21 -0700 Message-Id: <200804170654.m3H6sLkm006888@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP2: Some doubts Date: Wed, 16 Apr 2008 23:54:21 -0700 From: Bill Cheng Someone wrote: > 1. Do we need to wait for GET response? (should we show the command > prompt immediately after a get request has been sent?) You should wait for a GET response if you have flooded a GET request. But since you should only get one GET response, you don't have to wait for more GET responses once you get one response. > 2. For Search Request, Is it ok to convert SHA1 / Bitvector values to > their hex equivalents (%02x) before sending? (should we send 20 bytes > or 40) The spec is very clear about this. It says: ... If the search type is 2, the next field contains an exact SHA1 hash value (not hexstring-encoded). If the search type is 3, the next field contains a list of keywords (separated by space characters). Please follow the spec. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 16 23:29:24 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3H6TNSF015728 for ; Wed, 16 Apr 2008 23:29:23 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3H68Kvq005778 for ; Wed, 16 Apr 2008 23:08:20 -0700 Message-Id: <200804170608.m3H68Kvq005778@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP2: Some doubts Date: Wed, 16 Apr 2008 23:08:20 -0700 From: Bill Cheng SOmeone wrote: > I have some doubts: > > Q1. Related to Search: > The spec says: > > Responses to a search request should be presented as follows. A > response should start with a number followed by its metadata. ..... > ... When the user presses ^C (Control-c) on the keyboard, the > displaying of search results is terminated (although the node may be > receiving more search results) and a command prompt should be > displayed. > > > Should search responses also be expected only upto time limit (1.2 x > MsgLifetime - similar to STATUS Responses) or should the command > prompt be be shown after a search > request only after the user presses Control-C. A little after MsgLifetime, it would not make sense if you don't terminate the search! (The reason the spec explained what to do in case of is that it may not be clear exactly what should be the correct behavior, so additional explanation is required.) > Q2. Related to GET > A copy of the retrieved file should be saved into the current working > directory so that the user can access it. If is not > specified, the filename to use should be the same as FileName in the > file metadata. If such a file already exists, you should ask the user > if the file should be replaced. > > Should the be the filename only in the cwd or should the > file be now known as on this node (change the metadata > before saving etc). Please do not change the file metadata because it's part of the file. > Q3a) Additional Notes on File Retrieval.. > > - When a node performs a get and the file happens to be in the mini > filesystem of this node, the following should happen. If the file is > in the permanent space, a copy of the file should be placed in the > node's current working directory. > > If node A sends a search request, should the search also be performed > for A's own (cache) files? (I mean, should responses to that > particular search be performed for its own files???) Yes. > Q3b) If the file is in the cache space, its status should be changed > so that it is "moved" to the permanent space. > Continuation of Q3a -> Is get performed on all its own files OR cache > files OR none of its own files? GET must also apply to the node which initiate the GET. Please remember that other than the fact that a cached file is subject to LRU, there is really no difference between cached or permanent files. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 16 08:07:44 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3GF7iWD030786 for ; Wed, 16 Apr 2008 08:07:44 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3GEkj0r011364 for ; Wed, 16 Apr 2008 07:46:45 -0700 Message-Id: <200804161446.m3GEkj0r011364@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: fp2 doubt Date: Wed, 16 Apr 2008 07:46:45 -0700 From: Bill Cheng Someone wrote: > I wonder if it's ok if I check that if the node that initiate get > message already have that file(same filename sha1 and nonce) in > its file system it will just copy the file from its filesystem to > working dir and not flooding any message at all. I don't recall the spec or the grading guidelines say anything about this. So, I guess it's okay (unless someone points it out to me that it's mentioned in the spec or the grading guidelines). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Apr 14 12:51:39 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3EJpc1a021057 for ; Mon, 14 Apr 2008 12:51:38 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3EJM1gi030360 for ; Mon, 14 Apr 2008 12:22:01 -0700 Message-Id: <200804141922.m3EJM1gi030360@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP2 - Grading Guideline doubt Date: Mon, 14 Apr 2008 12:22:01 -0700 From: Bill Cheng I just noticed that this was my mistake. It should be fixed now. Sorry about it! -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- Date: Sun, 13 Apr 2008 23:03:07 -0700 From: Bill Cheng To: cs551@merlot.usc.edu Subject: Re: FP2 - Grading Guideline doubt Someone wrote: > When I try to run (B) - (1) - (a) Mix of nodes: from the grading guideli nes, I get the following error. Am I missing something? > > > ~csci551b/bin/resetf2 > > ~csci551b/public/final2/scripts/setup-r789 $HOME > > This script assumes that you have already run > ~csci551b/bin/setupf2 once. > > Please also make sure that you are running this script > from ~/grading/guidelines. > > If you have not done the above, please press and exit n ow! > If you have, please proceed. > > Please enter the first port number assigned to you or type : 111 30 > sh: /home/scf-22/csci551b/grading/guidelines/final2/init-i.02: cannot ope n > sh: /home/scf-22/csci551b/grading/guidelines/final2/init-i.03: cannot ope n > sh: /home/scf-22/csci551b/grading/guidelines/final2/init-i.04: cannot ope n > sh: /home/scf-22/csci551b/grading/guidelines/final2/init-i.08: cannot ope n Right above (A) in the grading guidelines, it has the following lines: For students, make sure you have done the following at least once: mkdir ~/bin mkdir ~/tmp mkdir ~/tmp/final2 mkdir ~/grading mkdir ~/grading/guidelines cd ~/grading/guidelines ~csci551b/bin/setupf2 $HOME Have you done that (especially the last line)? -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 13 23:39:42 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3E6dgjD011634 for ; Sun, 13 Apr 2008 23:39:42 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3E6AAgI012914 for ; Sun, 13 Apr 2008 23:10:10 -0700 Message-Id: <200804140610.m3E6AAgI012914@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt regarding indexes Date: Sun, 13 Apr 2008 23:10:10 -0700 From: Bill Cheng Someone wrote: > I have a confusion regarding some requirements about the indexes > that we maintain to speed up the searches. > The specification says > > "Each node in a index structure references a data file by file > name." > > This file name reffers to the ACTUAL file name or the internal > filename that we have generated to store (with extentions .data) The internal filename. Actually, it should just be the file number. This way, you can easily appent ".data" for the file, ".meta" for the file metadata, etc. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 13 23:37:53 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3E6brvq011454 for ; Sun, 13 Apr 2008 23:37:53 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3E68Kvo012836 for ; Sun, 13 Apr 2008 23:08:20 -0700 Message-Id: <200804140608.m3E68Kvo012836@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt on certificates Date: Sun, 13 Apr 2008 23:08:20 -0700 From: Bill Cheng Someone wrote: > When a node decides to store a message, where > should the certificate for the > file be stored. Should we create a file to store the certificate For "1.data", you can store the certificate as "1.cert". -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 13 23:35:06 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3E6Z6oQ011265 for ; Sun, 13 Apr 2008 23:35:06 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3E65XOo012781 for ; Sun, 13 Apr 2008 23:05:33 -0700 Message-Id: <200804140605.m3E65XOo012781@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP2- Grading guidelines - Keywords in metadata Date: Sun, 13 Apr 2008 23:05:33 -0700 From: Bill Cheng Someone wrote: > Case of the keywords in metadata: > > In the metadata of (A) - (2) - (a), when user enters the command > [ store blondie1.mp3 1 title="Heart of Glass" ] , the keywords > are mentioned all in lower letters , however in (A) - (1) - (a) > for the metadata corresponding to [ store chess.jpg 1 > categories="audio mp3" artist="Blondie" ] the keyword Blondie is > shown "as is". Should we be displaying the keywords in the > metadata "as is" or in lower case? Either way would be fine. The important thing is to make sure that you use lowercase letters for calculating bit-vectors. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 13 23:32:40 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3E6Wesi011069 for ; Sun, 13 Apr 2008 23:32:40 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3E6376h012743 for ; Sun, 13 Apr 2008 23:03:07 -0700 Message-Id: <200804140603.m3E6376h012743@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP2 - Grading Guideline doubt Date: Sun, 13 Apr 2008 23:03:07 -0700 From: Bill Cheng Someone wrote: > When I try to run (B) - (1) - (a) Mix of nodes: from the grading guidelines, I get the following error. Am I missing something? > > > ~csci551b/bin/resetf2 > > ~csci551b/public/final2/scripts/setup-r789 $HOME > > This script assumes that you have already run > ~csci551b/bin/setupf2 once. > > Please also make sure that you are running this script > from ~/grading/guidelines. > > If you have not done the above, please press and exit now! > If you have, please proceed. > > Please enter the first port number assigned to you or type : 11130 > sh: /home/scf-22/csci551b/grading/guidelines/final2/init-i.02: cannot open > sh: /home/scf-22/csci551b/grading/guidelines/final2/init-i.03: cannot open > sh: /home/scf-22/csci551b/grading/guidelines/final2/init-i.04: cannot open > sh: /home/scf-22/csci551b/grading/guidelines/final2/init-i.08: cannot open Right above (A) in the grading guidelines, it has the following lines: For students, make sure you have done the following at least once: mkdir ~/bin mkdir ~/tmp mkdir ~/tmp/final2 mkdir ~/grading mkdir ~/grading/guidelines cd ~/grading/guidelines ~csci551b/bin/setupf2 $HOME Have you done that (especially the last line)? -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 13 19:40:40 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3E2dwAL022617; Sun, 13 Apr 2008 19:39:58 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3E2ARKG009227; Sun, 13 Apr 2008 19:10:27 -0700 Message-Id: <200804140210.m3E2ARKG009227@bourbon.usc.edu> To: cs551@merlot.usc.edu, cs530@merlot.usc.edu, cs558@merlot.usc.edu Subject: FWD: USC Programming Contest Date: Sun, 13 Apr 2008 19:10:27 -0700 From: Bill Cheng Hi everyone, Please see the information below regarding the USC Programming Contest. You are encouraged to participate if you have time! Please note that participating in this contest does *not* execuse you from turning in assignments on time (sorry)! -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- Date: Sun, 13 Apr 2008 18:22:36 -0700 (PDT) From: Sven Koenig To: chengw@usc.edu Cc: Sven Koenig Subject: USC Programming Contest USC Programming Contest Saturday, April 26, 2008 in RTH 105/115 practice (1-2pm) and contest (2-6pm) www.cs.usc.edu/contest organized by David Kempe, Sven Koenig and Upsilon Pi Epsilon from the USC Computer Science Department Have fun! Extend your resume! Show how smart you are! Impress your friends! Win cash and other prizes! Represent USC at the ACM Programming Contest! Do you like to solve problems like the following ones? To pay your friends a dollar, you can give them, for example, four quarters, or five dimes and ten nickels. Write a program that calculates how many different combinations of coins there are that make up a given amount of money from pennies, nickels, dimes and quarters. You are given the description of a lake as a polygon, as well as your own position in a boat somewhere on the lake. When you throw a rock in the water, the waves travel away from the impact point at a speed of 1 meter per second. Write a program to calculate the earliest time at which a wave will hit the shore. You are given the names of n people, who are to be divided into k alphabetical segments, for instance, to stand in line for registration at USC. An "alphabetical segment" consists of all people whose last name starts with a letter between two given endpoints (for example, D-H). Write a program that finds a division into k alphabetical segments such that each person is in exactly one segment, and the number of people in the largest segment is minimized. Participate in the USC Programming Contest! The contest is open to all undergraduate and graduate USC students, regardless of major, as long as they were born in 1985 or later. We use the USC Programming Contests to pick students that we train for the ACM Regional Programming Contest. Find out more by visiting the contest webpage www.cs.usc.edu/contest If you would like to keep updated about the contest, please sign up for our mailing list at groups.yahoo.com/group/usc-programmingcontest Please contact Prof. David Kempe (dkempe@usc.edu) with any questions and to register for the contest. Registration deadline is April 23, 2008. Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 13 17:42:01 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3E0g1ZU011641 for ; Sun, 13 Apr 2008 17:42:01 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3E0CUxN007151 for ; Sun, 13 Apr 2008 17:12:30 -0700 Message-Id: <200804140012.m3E0CUxN007151@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt regarding Cache in FP2 Date: Sun, 13 Apr 2008 17:12:30 -0700 From: Bill Cheng Someone wrote: > According to:: > > - *CacheProb* - When a node forwards a get response message from a > neighbor, it flips a coin (with this probability of getting a positive > outcome) to decide if it should cache a copy of it. If the result is > positive, a copy of the file is stored. The default value is 0.1. > - *StoreProb* - When a node receives a store request from a neighbor, > it flips a coin (with this probability of getting a positive outcome) to > decide if it should cache a copy of it. If the result is positive, a copy of > the file is stored. The node where this store request was originated > ignores this value. The default value is 0.1. > > I understand that,when a node initiates a store on its commandline,the file > by default is stored in its perm space. Correct. > If it is a forwarded store > request,then it sees store prob and if it has to store then the file is > stored in its permanent space. It should go into the cache space. I'm not sure why you've determined that it should go into the permanent space! > But according to your lecture notes on Final Proj 2, > > if a node did not initiate a STORE (but only cache a copy > of the file in a STORE message), the file goes into its cache > (cont...)....PG 14 of Lecture Notes Correct. > The above sentence now says that if a node gets a forwarded store it has to > store in its cache.What is the correct way? If a node is just forwarding a STORE request, it should put the file in its cache space, if it decides to keep a copy of it. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 13 17:36:56 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3E0auBn011220 for ; Sun, 13 Apr 2008 17:36:56 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3E07QxR007004 for ; Sun, 13 Apr 2008 17:07:26 -0700 Message-Id: <200804140007.m3E07QxR007004@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: Question regarding to delete command Date: Sun, 13 Apr 2008 17:07:26 -0700 From: Bill Cheng Someone wrote: > As specification, when a node receives delete command, it should > verify the digital signature of the sender who issues the delete > command. If a node find out that verification is failed, it should > not delete the file yet. However, Does this node have to forward > (flood) this delete command to other nodes? Yes. (It could be the case that the certificate file it got is bad.) > One more thing, it is obvious that we can check the permission to > delete some particular file at the sender. Each node maintains every > certificates of every files (indicate the creator of each files), and > hence it can figure out by itself that it should have the permission > to delete that file or not? I'm not sure I understand what you said. By "the sender", do you mean the node that initiate a DELETE message (i.e., the one where the user entered a DELETE command)? Why does this node needs (or want) to check permissions? This node may not even have the file it's trying to delete. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Apr 13 17:28:51 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3E0Spil010368 for ; Sun, 13 Apr 2008 17:28:51 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3DNxLLE006829 for ; Sun, 13 Apr 2008 16:59:21 -0700 Message-Id: <200804132359.m3DNxLLE006829@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: get responce Date: Sun, 13 Apr 2008 16:59:21 -0700 From: Bill Cheng Someone wrote: > in the test case A) 2) a) the 3 beacon network. node 3 performs a > search and gets 3 responses . Then the node does a get 1 now the > cached file should be stored in the permanent space. for get 2 > and get 3 the files received are stored in the permanent space. > > Now when search is performed shouldn't the result have 5 > responses and node 3 has 3 copies of the same file.. in the test > case its says thr should only be 3 responses When the node finally have received the file, from the file metadata in the message, you should be able to determine that you have exactly the same file (with the same FileName, SHA1, and nonce). You should not save a copy of the file in HomeDir/files. You do have to make a copy in the current working directory. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Apr 12 09:28:48 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3CGSmlQ002454 for ; Sat, 12 Apr 2008 09:28:48 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3CFxTZS024259 for ; Sat, 12 Apr 2008 08:59:29 -0700 Message-Id: <200804121559.m3CFxTZS024259@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: doubt regarding certificate Date: Sat, 12 Apr 2008 08:59:29 -0700 From: Bill Cheng Someone wrote: > i have some doubt regarding the certificates: > > for a particular node do we need to create the private and the public keys > only once? Everytime after -reset. > we need to have these keys before hand before starting the node or when the > node starts the private and the public keys are to be found? This is one of the first thing your node should do, after parsing startup configuration file. If you don't have the private key and public key certificate file, you should create it programmatically. Otherwise, don't create a new pair. > i tried writing the following command on the command prompt before starting > the node and it says "openssl: Command not found" > > openssl req -new -nodes -newkey rsa:1024 -x509 -subj > "/C=US/ST=CA/L=LA/O=USC/OU=Servant-` date > +%d%b%Y`-rootca/CN=`hostname`/emailAddress=` > /usr/ucb/whoami`@usc.edu" -keyout private.pem -out cert.pem > along with my hostname and email address = whoami > for the date i wrote 041208. Please see: http://merlot.usc.edu/cs551-s08/projects/final.html#openssl and click on the link regarding how to run openssl on nunki. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 11 15:22:31 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3BMMUSt001972 for ; Fri, 11 Apr 2008 15:22:30 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3BLrHJG008228 for ; Fri, 11 Apr 2008 14:53:17 -0700 Message-Id: <200804112153.m3BLrHJG008228@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 2 Date: Fri, 11 Apr 2008 14:53:17 -0700 From: Bill Cheng Someone wrote: > In the LRU method for replacing files in the cache, there can be > the following scenarios > > 1> If the size of the file to be stored is greater than the size > of the cache itself, then we cannot save the file at all. So in > this case we have to delete the incoming file without touching > any file in the cache, since even if remove all the files from > cache we will not have enough space to save the incoming file. Correct. > 2> If the size of the file to be stored is such that we have to > delete multiple least recently used files then we should delete > the files till we have sufficient space to save the incoming > files. So theoretically it is possible to have a case where the > size of the incoming file is such that we would have to delete > all the files from cache to save the incoming file. Correct. > Is this the correct way to interpret the LRU method of caching? Yes. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 11 12:51:44 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=AWL,BAYES_05 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3BJpi2S020930 for ; Fri, 11 Apr 2008 12:51:44 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3BJMWBu005570 for ; Fri, 11 Apr 2008 12:22:32 -0700 Message-Id: <200804111922.m3BJMWBu005570@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 2 Date: Fri, 11 Apr 2008 12:22:32 -0700 From: Bill Cheng Someone wrote: > When you say a file already happens to be in the file system, > then it means that the file that you want to get from another > node or the same node has the same content (and hence same SHA1 > value) and same version (i.e having same Nonce) as the file > already present in the file system. The paragraph you referred to previously was for a very special case where the file (with the given FileID) you are looking for is already on the node. Since we have anonymity in our network, we generally don't know where a file for a given FileID is stored. > Let us consider the following scenario. There are two nodes that > are connected to each other 1->2. Now node 1 does a store for a > particular file and now the file is present on both nodes having > the same SHA1 and Nonce. Now on node 1 you do a search with the > filename and we get two results - one for the file already in > node 1 and other for the file in node 2. Now on node 1 you do a > get for the file present on node 2. Since the file on node 2 is > exactly identical to the file already on node 1 what should we > do? You must retrieve the file from node 2 with the correct FileID. -- Bill Cheng // bill.cheng@usc.edu > ----- Original Message ----- > From: Bill Cheng > Date: Friday, April 11, 2008 11:51 am > Subject: Re: CS551 Final Project part 2 > To: cs551@merlot.usc.edu > > > Someone wrote: > > > > > According to the new requirement "When a node performs a get > and > > > the file happens to be in the mini file system of this node, > the > > > following should happen. If the file is in the permanent > space, a > > > copy of the file should be placed in the node's current > working > > > directory.". > > > > This is not a new requirement. The next paragraph was modified > > on 4/8/08 because it used to say something like "if there is > > enough space in *permanent storage*". Since we don't use > PermSize, > > it was changed to "if there is enough space in its > *filesystem*". > > > > > So we need to check both the SHA1 and the Nonce of > > > the received file with that already in the file system to > decide > > > if the files are the same, right? > > > > In a GET request message, there is only a FileID. So, as > > long as the FileID matches, you should return the file. > > -- > > Bill Cheng // bill.cheng@usc.edu > > > > Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 11 12:11:38 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3BJBcQV017379 for ; Fri, 11 Apr 2008 12:11:38 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3BIgPdS004889 for ; Fri, 11 Apr 2008 11:42:26 -0700 Message-Id: <200804111842.m3BIgPdS004889@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 2 Date: Fri, 11 Apr 2008 11:42:25 -0700 From: Bill Cheng Someone wrote: > According to the new requirement "When a node performs a get and > the file happens to be in the mini file system of this node, the > following should happen. If the file is in the permanent space, a > copy of the file should be placed in the node's current working > directory.". This is not a new requirement. The next paragraph was modified on 4/8/08 because it used to say something like "if there is enough space in *permanent storage*". Since we don't use PermSize, it was changed to "if there is enough space in its *filesystem*". > So we need to check both the SHA1 and the Nonce of > the received file with that already in the file system to decide > if the files are the same, right? In a GET request message, there is only a FileID. So, as long as the FileID matches, you should return the file. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 11 10:56:58 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3BHuwpd010909 for ; Fri, 11 Apr 2008 10:56:58 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3BHRkf4003335 for ; Fri, 11 Apr 2008 10:27:46 -0700 Message-Id: <200804111727.m3BHRkf4003335@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP2 doubt: Store/Get Date: Fri, 11 Apr 2008 10:27:46 -0700 From: Bill Cheng Someone wrote: > Consider the foll network setup: > A -> B -> C -> D -> E > > 1. What is the correct behavior for a node when it gets a store request? > a) Should it read the entire (large) file before forwarding it ? Yes. > This would make file transfers very slow as entire file would have to be > stored first before being forwarded ahead. Correct. Pipelining would be faster. But I think store-and-forwad is easier to implement and debug. > That would mean writing the > complete file to disk before it can be read > since the file size could be larger than the file_buffer we use in the > program. You need to do that anyway if a node will be keeping that file. When moving a file around in our network, you should assume that the file is larger than the maximum buffer size and always use a temporary file. > b) Should it forward the file as it is being received? You can do it but I'm concerned that it may be too complex to implement and debug! When it comes time to grade, it doesn't matter how much code you have implemented or how efficient your code may run *if* it works. If your code locks up often during grading, you won't get a good score. > if A is sending a file to B and B has started forwarding the file to C > and midway through the file transfer > between A and B, their link breaks. Then B would have nothing to send > to C but since C is expecting part of > the file, B cant send any other message except that file which it was > transferring before the link failure. > As C would consider any incoming message as part of the file. C would > thus timeout and the link would break. > And this could propagate to D as well if C had started transferring > the portion to D. This would be a problem! > 2.Continuing the above question: > a) In case 1a > If A initiated a get request for a file which is stored on E, and if > D's CacheProb is false, would D temporarily > store the entire file before forwarding it to C.? Yes. You always use a temporary file. After you have finished creating the file and obtained its content, you then decide what to do with it. (By the way, CacheProb is a number between 0 and 1.) > b) In case 1a: > What if the message cache gets erased before the file is transferred > over multiple hops? > (B's cache gets erased before C receives the (large) file from D) Hmm... That would be a problem! In this case, the node who originated the GET request may not get a chance to receive the file. > 3. Can we use the 1 byte Reserve field in the message header for the > second case of question 1 above? That byte should be zero. I guess no one would notice if you use a single bit in that field. But I think you are making your program unnecessarily complex. > 4. On every store request / get response - the file is stored locally > in the nodes working_directory and in the > nodes homedir/files directory (assuming store prob is true). Yes. (Again, StoreProb is a number between 0 and 1.) > Since we will be using the filename to store the file in the > current_working_directory, what if after I have first > stored a file "blondie.mp3" (in current working dir) , I get another > store request for a different file with the same file name. > Should I overwrite the first file with the new file? Yes. You should probably prompt the user first to see if it's okay to overwrite the file. I want to stress that it's important to make your code work with the grading guidelines. Also, although your network should not be too slow, you will not get extra credit if your network is superfast! And if you implement more than what's in the spec, you won't get extra credit for it either. (So, if you want to do extra stuff, it will be purely for personal satisfaction!) -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Apr 11 10:21:30 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3BHLUrS007717 for ; Fri, 11 Apr 2008 10:21:30 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3BGqIcN002632 for ; Fri, 11 Apr 2008 09:52:18 -0700 Message-Id: <200804111652.m3BGqIcN002632@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt about grading guidelines "Memory allocation" Date: Fri, 11 Apr 2008 09:52:18 -0700 From: Bill Cheng Someone wrote: > I have a small doubt about the memory allocation section of > grading guidelines. The guidelines say that: > > "if allocate very large buffer (or size of file)". > > Does this also include the metadata files or only specific to data files. This only concerns about data files. You can allocate buffer whose size equals the size of the metadata (which is not too big). I guess potentially, a malicious user can use a million keywords for a file. But we don't have to deal with malicious users for this project. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 10 23:31:08 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3B6V8Xa014010 for ; Thu, 10 Apr 2008 23:31:08 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3B620Ei024808 for ; Thu, 10 Apr 2008 23:02:00 -0700 Message-Id: <200804110602.m3B620Ei024808@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt Date: Thu, 10 Apr 2008 23:02:00 -0700 From: Bill Cheng Someone wrote: > Can i keep an array for the search....and modify my files accordingly..... Sure you can do it, but you should have a dynamic array. Please do *not* use a fixed size array for this because it's a bad habit! I'm not sure what you meant by "modify your files accordingly". -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 10 00:30:52 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3A7UqIr021951 for ; Thu, 10 Apr 2008 00:30:52 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3A71q0M030895 for ; Thu, 10 Apr 2008 00:01:52 -0700 Message-Id: <200804100701.m3A71q0M030895@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP2: Grading guideline - Bit vector Date: Thu, 10 Apr 2008 00:01:52 -0700 From: Bill Cheng Someone wrote: > Please verify the Bit vector genrated for the following command: > > Grading guideline (A) - Beacon-only networks > > 1(a) (+2 points) > > store chess.jpg 1 categories="audio mp3" artist="Blondie" > > Bit Vectors generated for keywords: categories audio mp3 artist > Blondie should be, > > store chess.jpg 1 categories="audio mp3" artist="Blondie" > 0100000000000000000200000000000000000000000000000000000000000000002000000000000000000000000000000000000000040000000000000000000100000000000000000000000000000040000000088000000000000000400000000000000000000000000000100000000000000000000000000000000000000000 > > However the grading guideline mentions it to be: > 0100000000000000000200000000000000000000000000000000000000000000002000000000000000000000000000000000000000048000000000000000000000000000000000000000000000000000000000088000000000000000400080000000000000000000000000100000000000000000000000000000000000000000 > > I guess, the bit vector in grading guideline corresponds to > keyword blondie and not Blondie. > store chess.jpg 1 categories="audio mp3" artist="blondie" > 0100000000000000000200000000000000000000000000000000000000000000002000000000000000000000000000000000000000048000000000000000000000000000000000000000000000000000000000088000000000000000400080000000000000000000000000100000000000000000000000000000000000000000 > > I have verified the same by adding the bitvector generated for > each of the above keywords from > http://merlot.usc.edu/cs551-s08/projects/bitvec/ > > Kindly verify if I am missing something. In the Mini Filesystem section of the spec, under Bit-vector, it states that you must convert all keywords to lowercase before you compute a bit-vector! I have also mentioned this during lectures. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 9 20:21:00 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3A3L0Si031751 for ; Wed, 9 Apr 2008 20:21:00 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3A2q1lg027371 for ; Wed, 9 Apr 2008 19:52:01 -0700 Message-Id: <200804100252.m3A2q1lg027371@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP2: status response doubt Date: Wed, 09 Apr 2008 19:52:01 -0700 From: Bill Cheng Someone wrote: > The status response message doesn't have the field 'status_type'. Now how > does a receiving node will come to know whether the reply is of 'STATUS > NEIGHBORS' or of 'STATUS FILES'. Without knowing about this, we can't decide > whether to write the response in a NAM file or dump the metadata in the > output file. > > I think, 'status_type' should be a part of status response message and a > node receiving the status message should extract the 'status_type' filed > from the status message and put it in the response message. > > Alternately, we can put a global flag while submitting a status files > request and checking it again after we receive a status response message. > But I don't find it a good idea because we are relying on the program rather > than the message. When your node has just flooded a request, you should keep some state information about what it's waiting for. I don't think there is anything wrong with this. If it's possible for the node to send two different types of status request messagess in parallel, then it would be nice to have a statue_type in the status response message (although it's still not necessary since the inner UOID in the reply message can be used to distinguish what the reply is for). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Apr 9 19:24:00 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m3A2O0GH026709 for ; Wed, 9 Apr 2008 19:24:00 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3A1t2Aj026416 for ; Wed, 9 Apr 2008 18:55:02 -0700 Message-Id: <200804100155.m3A1t2Aj026416@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 2 Date: Wed, 09 Apr 2008 18:55:02 -0700 From: Bill Cheng Someone wrote: > But why do we want a file to be stored into the current working directory? The way you should think about sv_node is that it is installed somewhere else (and probably have some kind of setuid privilege) and you, as a user, are just running it. It may even be the case that you don't even have access to HomeDir. So, when you use sv_node to find a file for you and download it, as far as sv_node is concerned, it needs to keep a copy of the file and serve it in the future according to the peer-to-peer contract. Since you may not even have access to HomeDir, it needs to save a copy of the file in the current working directory for you so you can access it. In a way, your web browser kind of work like this if you think about its cache. When you download, say, a .tar.gz file, it keeps a copy of the file in its cache and save a copy in a directory of your choosing. -- Bill Cheng // bill.cheng@usc.edu On Tue, Apr 8, 2008 at 11:10 PM, Bill Cheng wrote: > > Someone wrote: > > > When we get a file from the network using the Get message, then > > the file is stored in the current working directory of the node. > > For the node that initiated the GET, the file goes into two > places. First, it goes into $HomeDir/files in the permanent > space. If that worked, it is then copied into the current > working directory. > > > Then why are we sending the certificate file in the get response > > message, since a file stored in the current working directory of > > a node does not show up on search command results and hence > > cannot be deleted. Only files in 'files' directory of a node can > > be deleted and hence sending the certificate file as a part of > > get response message can be deleted. The metafile is sent for the > > SHA1 matching, but there is no use for the certificate file. > > It's for deleting the file in the $HomeDir/files directory. > When a node received a DELETE message, it should *not* > delete any files in the current working directory. > -- > Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Apr 8 23:39:36 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m396daCs019480 for ; Tue, 8 Apr 2008 23:39:36 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m396Aixb004372 for ; Tue, 8 Apr 2008 23:10:44 -0700 Message-Id: <200804090610.m396Aixb004372@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 2 Date: Tue, 08 Apr 2008 23:10:44 -0700 From: Bill Cheng Someone wrote: > When we get a file from the network using the Get message, then > the file is stored in the current working directory of the node. For the node that initiated the GET, the file goes into two places. First, it goes into $HomeDir/files in the permanent space. If that worked, it is then copied into the current working directory. > Then why are we sending the certificate file in the get response > message, since a file stored in the current working directory of > a node does not show up on search command results and hence > cannot be deleted. Only files in 'files' directory of a node can > be deleted and hence sending the certificate file as a part of > get response message can be deleted. The metafile is sent for the > SHA1 matching, but there is no use for the certificate file. It's for deleting the file in the $HomeDir/files directory. When a node received a DELETE message, it should *not* delete any files in the current working directory. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Apr 8 16:24:25 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m38NOOBk013512 for ; Tue, 8 Apr 2008 16:24:25 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m38MtZlX030663 for ; Tue, 8 Apr 2008 15:55:36 -0700 Message-Id: <200804082255.m38MtZlX030663@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt regarding temporary folder: HW4 Date: Tue, 08 Apr 2008 15:55:35 -0700 From: Bill Cheng Someone wrote: > According to the specifications, we are to delete the temporary file (if > created). Is it also then required to delete the $HOME/tmp directory if it > was created through the program? No! Please do not programmatically delete *any directory*! Please also do not do somethingn like: system("rm -f *"); You should know what you have generated and delete only those files that you have generated. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Apr 8 11:53:33 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m38IrXqC021984 for ; Tue, 8 Apr 2008 11:53:33 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m38IOkeo025328 for ; Tue, 8 Apr 2008 11:24:46 -0700 Message-Id: <200804081824.m38IOkeo025328@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Regarding mod operations in SHA1 and MD5 for calculation the bit vector Date: Tue, 08 Apr 2008 11:24:46 -0700 From: Bill Cheng Someone wrote: > I am not able to figure out how do I perform the modulo operation > on the MD5 and SHA1 values received for keywords. > I get these values in a unsigned character array. > Is there some direct method to perfrom the modulo operation or we > have to do manupalation at the bit level. Let's take MD5 as an example. You need the last 9 bits of MD5. A MD5 value is stored in an array of 16 bytes. Let's say it's in md5_buf: unsigned char md5_buf[16]; Then the last 8 bits is just md5_buf[15]. You still need one more bit. If md5_buf[14] is an even number, then that bit is zero. If md5_buf[14] is a odd number, then that bit is one (and you have to add 256 to md5_buf[15]). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Apr 8 11:47:56 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m38IluXG021396 for ; Tue, 8 Apr 2008 11:47:56 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m38IJ8Cx025196 for ; Tue, 8 Apr 2008 11:19:08 -0700 Message-Id: <200804081819.m38IJ8Cx025196@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: fp2 question Date: Tue, 08 Apr 2008 11:19:08 -0700 From: Bill Cheng Hi everyone, I just updated the grading guidelines and deleted things about PermSize. Sorry about the delay. I have also fixed the spec with regard to PermSize. There were also language regarding "run out of space in permanent storage" which I have also modified to say "run out of space in the filesystem" instead. Please see all changes marked with today's date. Please note that PermSize is obsolete. If you see that in a startup configuration file, you should just ignore it and not print an error message! The rule for INI files is that it's perfectly fine to have additional keys and sectons. You should just look for the ones your program need. -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- Date: Mon, 07 Apr 2008 12:42:18 -0700 From: Bill Cheng To: cs551@merlot.usc.edu Subject: Re: fp2 question Someone wrote: > I remember you told us in class that there is no perm space limit > but why in the spec there still is. Am I misunderstanding? Oops! I forgot to remove them. Thanks for letting me know. I need to go through the grading guidelines carefully to remove them. I'll get it done by the end of tonight. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Apr 8 11:43:34 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m38IhYYa021002 for ; Tue, 8 Apr 2008 11:43:34 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m38IElqY025094 for ; Tue, 8 Apr 2008 11:14:47 -0700 Message-Id: <200804081814.m38IElqY025094@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP2: delete query Date: Tue, 08 Apr 2008 11:14:47 -0700 From: Bill Cheng Someone wrote: > What do we mean by "content-addressable network"? I have got a rough idea > about the usage, but I am not sure. We have discussed in class last Tuesday. -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- From: Bill Cheng [mailto:william@bourbon.usc.edu] Sent: Monday, April 07, 2008 7:55 PM To: cs551@merlot.usc.edu Subject: Re: CS551: FP2: delete query Someone wrote: > In the delete message, why do we need to send the SHA1 value? We can find a > unique file using the FILENAME and NONCE fileds only. Since the only index structure you have on a node is based on either FileName, SHA1, or keywords, there may be no easy way to find a file with a given Nonce. So, FileName and SHA1 should be part of a DELETE message. Since we are doing a content-addressable network, if you want to get rid of something here, you should get rid of FileName and not SHA1. I agree that they are a bit redundant, but this gives you an option in deciding which index structure to use. It's not a very strong reason. > Are we supposed to calculate the SHA1 of the file and check with the given > SHA1 before deleting it??? You are suppose to check SHA1 of a file before keeping it in your mini filesystem. So, you don't need to check again when it's time to delete it. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Apr 7 20:23:37 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m383Nbjb003926 for ; Mon, 7 Apr 2008 20:23:37 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m382stqb010390 for ; Mon, 7 Apr 2008 19:54:55 -0700 Message-Id: <200804080254.m382stqb010390@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP2: delete query Date: Mon, 07 Apr 2008 19:54:55 -0700 From: Bill Cheng Someone wrote: > In the delete message, why do we need to send the SHA1 value? We can find a > unique file using the FILENAME and NONCE fileds only. Since the only index structure you have on a node is based on either FileName, SHA1, or keywords, there may be no easy way to find a file with a given Nonce. So, FileName and SHA1 should be part of a DELETE message. Since we are doing a content-addressable network, if you want to get rid of something here, you should get rid of FileName and not SHA1. I agree that they are a bit redundant, but this gives you an option in deciding which index structure to use. It's not a very strong reason. > Are we supposed to calculate the SHA1 of the file and check with the given > SHA1 before deleting it??? You are suppose to check SHA1 of a file before keeping it in your mini filesystem. So, you don't need to check again when it's time to delete it. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Apr 7 14:31:49 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m37LVnVj005384 for ; Mon, 7 Apr 2008 14:31:49 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m37L3962003884 for ; Mon, 7 Apr 2008 14:03:09 -0700 Message-Id: <200804072103.m37L3962003884@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Question regarding metadata Date: Mon, 07 Apr 2008 14:03:09 -0700 From: Bill Cheng Someone wrote: > I have a small confusion about the file metadata information. > The specs say that the Nonce field should have the same value as the SHA1. Same *length*, not same value. > 1) What is the SHA1 value, is this the MD5 value > (like we created in warmup 1) (size 16). There is a link to the OpenSSL SHA-1 page in the spec. It's very similar to MD5. We use the SHA1() in the GetUOID() function. To calculate SHA-1 hash of a file, you should use SHA1_Init(), SHA1_Update(), and SHA1_Final(). > 2) Shall the value of Nonce be 16 or 20 like in GetUOID(). Nonce should be 20 bytes. To create a nonce, you should just call GetUOID() with obj_type="nonce". -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Apr 7 14:03:39 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m37L3dM4002834 for ; Mon, 7 Apr 2008 14:03:39 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m37KYxv8002391 for ; Mon, 7 Apr 2008 13:34:59 -0700 Message-Id: <200804072034.m37KYxv8002391@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: fp2 question Date: Mon, 07 Apr 2008 13:34:59 -0700 From: Bill Cheng Someone wrote: > Can I store cache file in "files" folder and use same sequence as > perm file? Sure! > I want to threat both cache and perm the same way and have > another file to indicate which one is perm or cache. Yes, I think it's reasonable to do it this way. This *other* file to indicate which one is perm or cache can basically be the LRU list! If a file is not on the LRU list, then it's in permanent storage. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Apr 7 13:10:58 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m37KAw0I030573 for ; Mon, 7 Apr 2008 13:10:58 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m37JgI1f000891 for ; Mon, 7 Apr 2008 12:42:18 -0700 Message-Id: <200804071942.m37JgI1f000891@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: fp2 question Date: Mon, 07 Apr 2008 12:42:18 -0700 From: Bill Cheng Someone wrote: > I remember you told us in class that there is no perm space limit > but why in the spec there still is. Am I misunderstanding? Oops! I forgot to remove them. Thanks for letting me know. I need to go through the grading guidelines carefully to remove them. I'll get it done by the end of tonight. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Apr 5 09:39:01 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m35Gd13T012918 for ; Sat, 5 Apr 2008 09:39:01 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m35GAd2g027627 for ; Sat, 5 Apr 2008 09:10:39 -0700 Message-Id: <200804051610.m35GAd2g027627@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP2 question Date: Sat, 05 Apr 2008 09:10:39 -0700 From: Bill Cheng Someone wrote: > Can you give me some hint how to serialize object into files in case of > "kwrd_index", "name_index" and "sha1_index"? > Are there a way to do this automatically or I have to write my own > function to serialize and deserialize obj (I use C++ map). I'm not familiar with C++ map, but I assume that there is a way to enumerate all objects in a map. So, you can call this enumeration function. In the callback, you can write out: key,value\n to a index file (assuming you are using text files). The value here would be just a numeric value which corresponds to a file's number in $HomeDir/files. It would be easier to discuss this over office hours if this is not clear. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Apr 3 23:37:53 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m346brGf023250 for ; Thu, 3 Apr 2008 23:37:53 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m3469gZJ026049 for ; Thu, 3 Apr 2008 23:09:42 -0700 Message-Id: <200804040609.m3469gZJ026049@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 2 Date: Thu, 03 Apr 2008 23:09:42 -0700 From: Bill Cheng Someone wrote: > In the store message, when a node that is receiving a store > message decides not to store the file, then it will not be in a > position to forward the file too since it will not have a copy of > the file. So when a node receives a store request from a neighbor > it will flip a coin to decide whether to cache the file or not. > If it does not cache the file then it will not forward the > request too right? Correct. But it's not because "it will not have a copy of the file". Even if a node decides not to keep a file, it can still keep it in temporary storage. The real reason it should not forward the file if it decides not to keep a file is because that's how we have specified our probabilistic flooding algorithm. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 31 13:46:18 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2VKkIqV002849; Mon, 31 Mar 2008 13:46:18 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2VKIZan024482; Mon, 31 Mar 2008 13:18:35 -0700 Message-Id: <200803312018.m2VKIZan024482@bourbon.usc.edu> To: cs558@merlot.usc.edu, cs530@merlot.usc.edu, cs551@merlot.usc.edu Subject: Faculty candidate talk tomorrow on Distribute Systems / Security / OS Date: Mon, 31 Mar 2008 13:18:35 -0700 From: Bill Cheng Hi everyone, Below is a talk announcement for tomorrow from 3:30pm-5pm in SSL 150. You are encouraged to attend the talk. (Attendence is optional and you will not be responsible for the talk material as far as grading/exam is concerned.) -- Bill Cheng // bill.cheng@usc.edu ---------------------------------- cut here ---------------------------------- Title: Securing the Web With Decentralized Information Flow Control Speaker: Maxwell Krohn, MIT, CS and AI Laboratory Venue: SSL 150 Date & Time: 3:30-5:00pm, Tuesday April 1st The recent successes of server-side applications (e.g. Google and Facebook applications) hint that tomorrow's computing platform might not be the local desktop but rather the extensible remote Web site. Unfortunately, these new server-side platforms, built on conventional operating systems, are committing the same security mistakes already ossified in today's insecure desktops. In this talk, I will discuss how to secure both today's Web sites and tomorrow's Web computing platforms with a new OS technique called Decentralized Information Flow Control (DIFC). A DIFC system tracks the flow of secret data as it is copied from file to file and communicated from process to process. In the end, the OS lets modules known as "declassifiers" legislate policies for secret data exiting to the network. DIFC provides better security than standard OSes because it allows developers to concentrate security-critical code in small, audit-friendly declassifiers, which remain small and contained even as the overall system balloons with new features. This talk presents DIFC, an implementation of DIFC for Linux, and a case study of a complex, popular open-source application (MoinMoin Wiki) secured with DIFC. MoinMoin is a prototype for more ambitious and general work to come, such as a novel Web-based application platform with encouraging security guarantees. Joint work with: Micah Brodsky, Natan Cliffer, Petros Efstathopoulos, Cliff Frey, Eddie Kohler, David Mazieres, Robert Morris, Frans Kaashoek, Steve VanDeBogart, Mike Walfish, Alex Yip, David Ziegler Bio: Maxwell Krohn is a PhD candidate in Computer Science at MIT. He received his BA from Harvard in 1999, and was a staff research scientist at NYU from 2002-2003. In between, he co-founded and co-built several community Web sites, some vintage (TheSpark.com), others live and kicking (SparkNotes.com and OkCupid.com). His research interests are in operating systems, distributed systems and security. Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 31 12:02:25 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2VJ2PBQ025991 for ; Mon, 31 Mar 2008 12:02:25 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2VIYhkh022095 for ; Mon, 31 Mar 2008 11:34:43 -0700 Message-Id: <200803311834.m2VIYhkh022095@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Questions on FP part 2 Date: Mon, 31 Mar 2008 11:34:43 -0700 From: Bill Cheng Someone wrtoe: > I have serveral questions regarding the final project part2: > (1) Do we assume that the "files" directory exists on every sv > node, so that we don't need to check and create one if it doesn't > exist? According to the grading guidelines, before running any tests, we run "~csci551b/bin/resetf2" which creates all the "files" subdirectories under HomeDir. But, if you are paranoid, you should call mkdir() if there is no "files" subdirectory under HomeDir. Please do *not* remove any subdirectories under HomeDir even if you start your node with "-reset". When you run with "-reset", you should remove all the files you have created programatically but not directories. > (2) Is there any requirement for where to hold the cached files? > I am considering to put cached files in memory rather than in > disk, and it implies that all cached files would disappear after > the node shutdown or crashed. So I wonder are we required to keep > the cached files available when the node starts up (without > -reset) again? Cached files must be stored somewhere under the $HomeDir/files directory. After a node dies and get restarted again, all the cached files must be there. In the grading guidelines, this is one of the first things that gets tested. Please see (A.1.b). Please note that this is an important behavior for a peer-to-peer system, so please do not keep cached files in memory. (Also, cached files can be large, so you should never keep them in memory.) I have just added this requirement in the minus points section of the grading guidelines. Now it contains the following line: Cached files not persistent after shutdown : -20 points > (3) Can I put the .pem file along with .data and .meta under the > "files" directory? Yes! > Are there any restrictions on what should go into the "files" directory? No. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 30 19:22:38 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2V2McuO001061 for ; Sun, 30 Mar 2008 19:22:38 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2V1t18s004358 for ; Sun, 30 Mar 2008 18:55:01 -0700 Message-Id: <200803310155.m2V1t18s004358@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CSCI551 Date: Sun, 30 Mar 2008 18:55:01 -0700 From: Bill Cheng Someone wrote: > I am facing a problem during the unlinking (deletion of the data file). > Whenever the client restarts, unlinks the neighbour list file( u have used > the accept system call) and sends a connect request to the server, the > server accept fails showing an error "too many files open". I'm confused about what you are asking. If you want to unlink a file, you should just call unlink(). I'm not understanding what is unlinking the init_neighbor_list file has anything to do with accept(). For part (2), you don't need to remove the init_neighbor_list file. May be you should come to office hours to talk about this. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 27 19:54:53 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2S2srwO001109 for ; Thu, 27 Mar 2008 19:54:53 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2S2Rexx030620 for ; Thu, 27 Mar 2008 19:27:40 -0700 Message-Id: <200803280227.m2S2Rexx030620@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Issue with pthread_cancel() Date: Thu, 27 Mar 2008 19:27:40 -0700 From: Bill Cheng Someone wrote: > I am facing a problem with pthread_cancel() on which I need some > help from you. > > In my code I start a few threads initially and all wait at > pthread_cond_wait() and signalled using pthread_cond_broadcast(). > Everyting is working fine untill I call a pthread_cancel() to > terminate one of the waiting threads. > > After canceling the thread, when I try calling the routing : > > pthread_mutex_lock(&mNewMessageMutex); > pthread_cond_broadcast(&mNewMessageCond); > pthread_mutex_unlock(&mNewMessageMutex ); > > the application hangs. On debuging I find that the calling thread > is unable to aquire a lock on > pthread_mutex_lock(&mNewMessageMutex); > > Please sugest what should I try to resolve this. I think you've killed your thread whene it has a lock on the mutex. This can happen if you did not set the "cancel type" and the "cancel state" properly. If you do "man pthread_cancel", there should be enough information there about the following two functions: pthread_setcancelstate() pthread_setcanceltype() You should call these functions so that you can only kill the thread when it's at a "cancellation point". Another way is to never cancel a thread! (That's what I would do since this follows the "structured programming principle".) You can always set a global flag, wake up a thread, and ask it to terminate on its own by executing the return statement of the thread creation function. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 26 21:35:03 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2R4Z2Fb005847 for ; Wed, 26 Mar 2008 21:35:02 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2R47w7q002264 for ; Wed, 26 Mar 2008 21:07:58 -0700 Message-Id: <200803270407.m2R47w7q002264@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 2 Date: Wed, 26 Mar 2008 21:07:58 -0700 From: Bill Cheng Someone wrote: > In the 'store' user command and the file metadata are the > keywords only separated by spaces? Basically yes. > Or is it that like C, C++ in > order to show the continuation of a command on the next line we > need to incorporate the backslash. Well, it's not C/C++ but more of a UNIX convention. You can implement this if you want (and document it). As you can see from the grading guidelines, everything fits in a line when it comes to using the STORE command. So, we will only test with putting everything in a (possibly very long) line. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 26 12:04:07 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2QJ46nK020249 for ; Wed, 26 Mar 2008 12:04:06 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2QIb5Dm023715 for ; Wed, 26 Mar 2008 11:37:05 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m2QIb5f9023714 for cs551@merlot; Wed, 26 Mar 2008 11:37:05 -0700 Date: Wed, 26 Mar 2008 11:37:05 -0700 From: Bill Cheng Message-Id: <200803261837.m2QIb5f9023714@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Final project part (2) grading guidelines have been posted! Hi everyone, I've just posted the grading guidelines for part (2) of the final project. Please take a look. If you did not do well with part (1), one way you can start part (2) is to start with the first case in the part (2) grading guidelines. The first step would be to fix your code so that without entering any part (2) commands, your network is stable and can handle part (1) commands with the configurations in the part (2) grading guidelines. If you have questions or see bugs in the grading guidelines, please let me know! Thanks! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Mar 25 11:12:21 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2PICL5l014982 for ; Tue, 25 Mar 2008 11:12:21 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2PHjSbK029199 for ; Tue, 25 Mar 2008 10:45:28 -0700 Message-Id: <200803251745.m2PHjSbK029199@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP1 doubt: how to see *.out using nam Date: Tue, 25 Mar 2008 10:45:28 -0700 From: Bill Cheng Hi everyone, As it is emphasized in the spec, it is imperative that your status command works! This includes getting the output to work with ~csci551/nam. -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- Date: Tue, 25 Mar 2008 10:43:54 -0700 From: Bill Cheng To: cs551@merlot.usc.edu Subject: Re: FP1 doubt: how to see *.out using nam Someone wrote: > I changed the file name to "status.nam", then I can see it in nam > console. > > nam console only takes *nam *nam.gz. How are we supposed to view > an *.out file? If you just run "~csci551/nam", you can select Open from the File menu. Then you can just type "status.out" in the File Name box and click on Open. Or, you can just do: ~csci551/nam status.out -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Mar 25 11:10:47 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2PIAlgB014797 for ; Tue, 25 Mar 2008 11:10:47 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2PHhsXW029159 for ; Tue, 25 Mar 2008 10:43:54 -0700 Message-Id: <200803251743.m2PHhsXW029159@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP1 doubt: how to see *.out using nam Date: Tue, 25 Mar 2008 10:43:54 -0700 From: Bill Cheng Someone wrote: > I changed the file name to "status.nam", then I can see it in nam > console. > > nam console only takes *nam *nam.gz. How are we supposed to view > an *.out file? If you just run "~csci551/nam", you can select Open from the File menu. Then you can just type "status.out" in the File Name box and click on Open. Or, you can just do: ~csci551/nam status.out -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Mar 25 11:08:01 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2PI81lx014430 for ; Tue, 25 Mar 2008 11:08:01 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2PHf8nj029106 for ; Tue, 25 Mar 2008 10:41:08 -0700 Message-Id: <200803251741.m2PHf8nj029106@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP1 doubt: how to see *.out using nam Date: Tue, 25 Mar 2008 10:41:08 -0700 From: Bill Cheng Someone wrote: > I don't know how to see *.out using nam. Here's what I do: > > I'm using X-win, I typed following command, under the same > directory when status.out is present: > > ~csci551/ns > ~csci551/nam > > then a graphic interface pops out, then I type > > nam status.out > > in previous command line interface. Nothing happens... > > I download status.out directly from project homepage, > > How am I supposed to make nam work? You should do "nam status.out" before you have a graphics interface. If "nam" is not in your PATH environment variable, you should do the following directly: ~csci551/nam status.out -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 24 23:42:46 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2P6gkai017732 for ; Mon, 24 Mar 2008 23:42:46 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2P6FvR8016950 for ; Mon, 24 Mar 2008 23:15:57 -0700 Message-Id: <200803250615.m2P6FvR8016950@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Late submission for CS551 FP1 Date: Mon, 24 Mar 2008 23:15:57 -0700 From: Bill Cheng Someone wrote: > Does this project also have same policy as of warm up project1 and 2 for > late Submission ?. How much marks will be deducted in case if we submit > after 1 day of the deadline. Please see: http://merlot.usc.edu/cs551-s08/projects.html#late -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 24 23:09:02 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2P692mn014552 for ; Mon, 24 Mar 2008 23:09:02 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2P5gDjj015604 for ; Mon, 24 Mar 2008 22:42:13 -0700 Message-Id: <200803250542.m2P5gDjj015604@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 : FP1 Case C.3.b doubt Date: Mon, 24 Mar 2008 22:42:13 -0700 From: Bill Cheng Someone wrote: > In Case C.3.b wherein you shutdown the node *00 and *04 then the system > should rejoin and wait till autoshutdown time(180 secs) and then shutdown > completely . But the system does not shutdown even after 15 min. Could this > be because of the load on nunki? Don't know! Most likely, it's a bug. > Also to be noted that the other > autoshutdown cases do work for the same system. May be it's a timing bug that only happens when a certain condition is met. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 24 23:05:28 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_40,WEIRD_QUOTING autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2P65RUO014197 for ; Mon, 24 Mar 2008 23:05:28 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2P5ccDI015537 for ; Mon, 24 Mar 2008 22:38:39 -0700 Message-Id: <200803250538.m2P5ccDI015537@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt regarding Fp1 Date: Mon, 24 Mar 2008 22:38:38 -0700 From: Bill Cheng Someone wrote: > I had one question regarding test case : > regarding test case :: (2)-a --> C8 -a > > Actually test case mentions topology n07*<-n04<-n09<-n06 > > consider following scenario : > n07 is up. > n04 sends join request > simultaneously n09 sends join request > now n04 is still waiting for join to timeout and it cant process any other > request as (node is not officially ""up"" ) > n09 has already executed JoinTimeout ..so it receives only 1 entry from > n07. > so init_neighbor_list contains of n09 has 1 entry of n07 and not of n04( as > expected in test case ) .. > so is this valid/expected behavior ? or there is some problem .. You need to determine if this is a valid behavior or not! > also could you please give us some info abt how much delay we can assume for > multiple join request to a node. I don't know what you meant by "how much delay". I don't see much difference between a single join request and multiple join requests. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 24 22:56:55 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2P5utEb013485 for ; Mon, 24 Mar 2008 22:56:55 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2P5U6LX015353 for ; Mon, 24 Mar 2008 22:30:06 -0700 Message-Id: <200803250530.m2P5U6LX015353@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Final project part (1) deadline extended by 1 day! Date: Mon, 24 Mar 2008 22:30:06 -0700 From: Bill Cheng Someone wrote: > I cant see any changes in the grading guide line. I would be > obliged if you could let me know which part have u changed. Sorry, but I cannot do that! You can do a "diff" between the copy you have and what's in ~csci551b/public/final1/grading_guidelines. The latest grading guidelines has 1,207 lines. If that's not what you have, you may need to purge cached files in your browser. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 24 08:11:05 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2OFB5wC032225 for ; Mon, 24 Mar 2008 08:11:05 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2OEiL2B000331 for ; Mon, 24 Mar 2008 07:44:21 -0700 Message-Id: <200803241444.m2OEiL2B000331@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt regarding Fp1 Date: Mon, 24 Mar 2008 07:44:21 -0700 From: Bill Cheng Someone wrote: > You mentioned that you should wait for approx 1.2 x MsgLifetime for status > responses. > > But in the test cases, after a "status neighbors" command, for ex: (B) 1 > (c): > > > in 3rd window, do: > ./sv_node r5-n09.ini > (this node also autoshutdown in 60 seconds, InitNeighbors=2, > JoinTimeout=5, KeepAliveTimeout=10, MinNeighbors=2) > (this node should put both nodes in its init_neighbor_list) > (wait 10 seconds, at the commandline prompt, type > "status neighbors 5 09.nam", look at "09.nam") > > in 3rd window (node *09), wait 5 seconds and type "shutdown" > in 2nd window (node *04), wait 5 seconds and type "shutdown" > in 1st window (node *00), wait 5 seconds and type "shutdown" > > > Should I wait for the status neighbors response before I type shutdown or > Should I press shutdown about 5 seconds after typing the "status' command. > > If I should wait, then I end up waiting for a long time (1.2 x MsgLifeTime) > before the response is received. Since everything is running on nunki, there should be no need to wait for a long time. Waiting for *enough* time and pressing Also, > If Control-C is pressed or if the node shuts itself down or receives a > sigterm, should a nam file > be written for whatever responses that node has recd till that time or > should it just quit? Pressing should be the same as timing out early. Therefore, you should produce the nam file. Also, you are suppose to create the nam file on-the-fly, so this should not be a problem at all! -- Bill Cheng // bill.cheng@usc.edu On Mon, Mar 17, 2008 at 4:02 PM, Bill Cheng wrote: > Someone wrote: > > > I have doubt regarding how status command works. > > > > We have to flood status message until TTL is 0. > > > > Now , sending node has to wait for all the status responses to each of > nodes > > received?. How will the sender node know , how long to wait for status > > messages or till how many status messsages it has to receive? > > You can use a small multiple of MsgLifetime to time this out. > The assumption is the after MsgLifetime, nodes in the network > will no longer able to forward responses back to you, so it's > reasonable to timeout. You can use 1.2 as the multiple (I > think that's the value I mentioned in class). > -- > Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 24 08:04:25 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2OF4PDY031678 for ; Mon, 24 Mar 2008 08:04:25 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2OEbfPS032682 for ; Mon, 24 Mar 2008 07:37:42 -0700 Message-Id: <200803241437.m2OEbfPS032682@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: Final1 - Information printing Date: Mon, 24 Mar 2008 07:37:41 -0700 From: Bill Cheng Someone wrote: > Should it be better if we print current state information of the > program such as "System is restarted", "successfully connected to > node XX", or "waiting for X seconds to JOIN" to STDOUT to help the > grader understand the current program status? or we just leave the > STDOUT clear and print these information in the log file? If you want to print these to stdout, you must make sure that they do not interfere with the user interface. This can be a little tricky. You should definitely print additional useful information into the log file, especially if you think it will help with grading! But make sure you start these lines with "//". -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:57:01 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=AWL,BAYES_20 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6v1PD020052 for ; Sun, 23 Mar 2008 23:57:01 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O6UKmf024084 for ; Sun, 23 Mar 2008 23:30:20 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m2O6UKC1024083 for cs551@merlot; Sun, 23 Mar 2008 23:30:20 -0700 Date: Sun, 23 Mar 2008 23:30:20 -0700 From: Bill Cheng Message-Id: <200803240630.m2O6UKC1024083@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Final project part (1) deadline extended by 1 day! Hi Everyone, In my message with timestamp "Thu 20 Mar 22:16" (see below), I mentioned that I've posted some changes to the grading guidelines. I just noticed that I didn't post the changes! So, I'm extending the final project part (1) deadline by 1 day to 11:45pm on Tue, 3/25/2008. By the way, I will be talking about final project part (2) on Tuesday's lecture (on 3/25/2008). -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- Date: Thu, 20 Mar 2008 22:16:29 -0700 From: Bill Cheng To: cs551@merlot.usc.edu Subject: Minor changes to the grading guidelines... Hi everyone, I add a few tests to the grading guidelines. Basically, these tests try to see if you update the in-memory list of neighbors properly. Before, in the grading guidelines, there were a few places where I shutdown a list of nodes in sequence. I've added some "status neighbors" commands after I shutdown some of the nodes to check if the topology is correct, before I shutdown the rest of the nodes. The bottomline is that these do not really change anything. Just a few more tests. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:46:07 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6k7jb019108 for ; Sun, 23 Mar 2008 23:46:07 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O6JQ71023913 for ; Sun, 23 Mar 2008 23:19:26 -0700 Message-Id: <200803240619.m2O6JQ71023913@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: FP1 - Timeout for MinNeighbors Date: Sun, 23 Mar 2008 23:19:26 -0700 From: Bill Cheng Someone wrote: > Is there a timeout for the MinNeighbors value that we need to > check when after a non-beacon node starts waiting for the hello > responses. If there isn't, can you suggest us one? I assume you mean a "join" response (and not a "hello" response). Please use JoinTimeout. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:44:05 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6i5Sh018925 for ; Sun, 23 Mar 2008 23:44:05 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O6HOsJ023881 for ; Sun, 23 Mar 2008 23:17:24 -0700 Message-Id: <200803240617.m2O6HOsJ023881@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt in Final1 Date: Sun, 23 Mar 2008 23:17:24 -0700 From: Bill Cheng Someone wrote: > I am getting a seg fault and when I use gdb and back trace it says--- > corrupted stack > Can you please suggest me something. This would be a typical memory corruption bug. Please see: http://merlot.usc.edu/cs551-s08/projects.html#segfault Sorry that I don't have a better suggestion. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:42:03 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6g3Eo018742 for ; Sun, 23 Mar 2008 23:42:03 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O6FNg5023849 for ; Sun, 23 Mar 2008 23:15:23 -0700 Message-Id: <200803240615.m2O6FNg5023849@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt on final project part1 Date: Sun, 23 Mar 2008 23:15:23 -0700 From: Bill Cheng Someone wrote: > I wanted to know if we can use posix 1b timers > for managing the timeouts for our project. > Are there any issues in using posix 1b timers. I have no idea what's POSIX 1b timers. If there are readily available on nunki in a system directory, there is absolutely no problem. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:40:02 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6e2TX018436 for ; Sun, 23 Mar 2008 23:40:02 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O6DLOo023810 for ; Sun, 23 Mar 2008 23:13:21 -0700 Message-Id: <200803240613.m2O6DLOo023810@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Regarding final project. Date: Sun, 23 Mar 2008 23:13:21 -0700 From: Bill Cheng Someone wrote: > The doubt scenario is: > > During the connection setup phase , A and B form 2 connections. > > When A receives the HLLO msg and decides that it has to tear > down one on of the connections , does it send a notify on this > connection ? If it has sent a HELLO message, it should send NOTIFY. > If yes , then on the receiving this notify , does B need to send > check on other connections ? No. Because this node still has a connection with the other node. It only dropped a "duplicate" connection. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:34:52 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6Yqkg018013 for ; Sun, 23 Mar 2008 23:34:52 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O68Bm2023738 for ; Sun, 23 Mar 2008 23:08:11 -0700 Message-Id: <200803240608.m2O68Bm2023738@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: status msg Date: Sun, 23 Mar 2008 23:08:11 -0700 From: Bill Cheng Someone wrote: > if the topology is A--B--C and we set TTL of status message sent > from A to 1. > How the nam file look like? > A--B or A--B--C ??? The status request will reach node B and node B will report that it has neighbors A and C. So, the network should look like A--B--C. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:31:35 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6VYXM017646 for ; Sun, 23 Mar 2008 23:31:35 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O64sgi023676 for ; Sun, 23 Mar 2008 23:04:54 -0700 Message-Id: <200803240604.m2O64sgi023676@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: About startup Date: Sun, 23 Mar 2008 23:04:54 -0700 From: Bill Cheng Someone wrote: > Do we have to implement the status part--- > status files for part 1 No. As you can see in the grading guidelines, there is no "status files ..." -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:30:31 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6UVp6017633 for ; Sun, 23 Mar 2008 23:30:31 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O63oik023554 for ; Sun, 23 Mar 2008 23:03:50 -0700 Message-Id: <200803240603.m2O63oik023554@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 1 Date: Sun, 23 Mar 2008 23:03:50 -0700 From: Bill Cheng Someone wrote: > According to this new requirement 'The log file should be deleted > only when a node is started with the -reset command line > option.'. This creates a problem. Suppose I have a beacon node > and one regular node. Initially there are no files in the home > directory of this regular node. So the first time the regular > node is started it joins the network and this is put into the log > file. Then after 5 seconds I do a shutdown. Then I again start > the regular node without the '-reset ' option since I want the > regular node now to participate in the network (and not join the > network). But as the '-reset' option is not used, the log > messages get appended to the log file that already exists. This > is the same scenario as in the test case (B) 1 b. The grader has > to verify that the log file does not contain any Join messages. > But the since we appended the new log messages to the earlier log > file, the join messages received and sent from the earlier test > still exists. > > So this requirement is not suitable, when a node is shutdown > completely and then restarted by the user by typing the command > 'sv_node b0...'. Could you please clarify the requirement. This should be considered as bugs in the grading guidelines. Before starting test (B.1.b), the log files should be removed. I've updated the grading guidelins. There is a similar situation in (C.1.b). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:20:02 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6K2Bn016483 for ; Sun, 23 Mar 2008 23:20:02 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O5rL2U023303 for ; Sun, 23 Mar 2008 22:53:21 -0700 Message-Id: <200803240553.m2O5rL2U023303@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 1 Date: Sun, 23 Mar 2008 22:53:21 -0700 From: Bill Cheng Someone wrote: > The test cases in the grading guidelines asks us to verify the > port numbers messages are received from. Could you let me know exactly where it says that? May be I'm misunderstanding what you meant. Your program should *never* check to make sure that the port numbers are the ones assigned to you! Just use the port numbers given in the startup configuration files. But, when you are running your code, you should only use port numbers assigned to you so you do not mess up other students' tests. > However since we are > running the scripts locally from our account, the log files would > now contain our port numbers. Exactly! > Could you please give us the > starting port number of the grader's account (the one for which > the test cases have been designed). We can use the offset from > that and add it to our starting port number to make verify the > test cases with our port numbers. Please do not do that! You should make sure that your code will work with any legitimate port numbers! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 23 23:14:58 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2O6Ewig016076 for ; Sun, 23 Mar 2008 23:14:58 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2O5mHsd023195 for ; Sun, 23 Mar 2008 22:48:17 -0700 Message-Id: <200803240548.m2O5mHsd023195@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Fwd: CS551: grading guideline question Date: Sun, 23 Mar 2008 22:48:17 -0700 From: Bill Cheng Someone wrote: > I understand that "servant.log" should be like you said. However, the > question below, I meant "00.out" which is NAM file. What would the > topology of this beacan node see? I'm sorry, I cannot answer questions like this! I think you should be able to figure it out! -- Bill Cheng // bill.cheng@usc.edu Begin forwarded message: > From: Bill Cheng > Date: March 22, 2008 7:14:13 AM PDT > To: cs551@merlot.usc.edu > Subject: Re: CS551: grading guideline question > > Someone wrote: > >> I have a question regarding to a grading guideline: >> >> ===================================================================== >> === >> ============================ >> (d) (+3 points) a single beacon node with a regular node joining the >> network, the join would fail because there is not enough >> neighbors >> >> Restart the beacon node in the first window as follows: >> ~csci551b/bin/resetf1 >> /bin/rm ??.out >> ./sv_node r5-n00.ini >> (wait till next node get started, then wait 10 >> seconds, >> at the commandline prompt, type >> "status neighbors 1 00.out", look at >> "00.out") >> >> in 2nd window, do: >> ./sv_node r5-n05.ini >> (this node has InitNeighbors=3, join should fail >> after 5 >> seconds, this node should give up and die) >> ===================================================================== >> === >> ============================ >> >> What should go into "00.out" file? nothing (since node n05 just tried >> to JOIN to n00 and later failed - not consider as a connection >> establishment) or there should be a topology (2 nodes and a link of a >> single beacon node with a regular node)? > > JOIN requests should be logged. > > If there are other nodes the beacon node is connected to, > it will have to flood the JOIN request. So, it should > have a record of where the request came from. > -- > Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 22 07:52:15 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2MEqFSb001118 for ; Sat, 22 Mar 2008 07:52:15 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2MEPmWV029753 for ; Sat, 22 Mar 2008 07:25:48 -0700 Message-Id: <200803221425.m2MEPmWV029753@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: About startup Date: Sat, 22 Mar 2008 07:25:48 -0700 From: Bill Cheng Someone wrote: > For a beacon node, if the init_neighbor_list file contains > non-beacon nodes, during startup, should a connection be > initiated by the beacon node to the non-beacon nodes as well? A beacon node should *never* initiate a connection to a non-beacon node! Also, a beacon node does *not* have a init_neighbor_list file! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 22 07:50:11 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=AWL,BAYES_20 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2MEoBhD000929 for ; Sat, 22 Mar 2008 07:50:11 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2MENhMM029693 for ; Sat, 22 Mar 2008 07:23:43 -0700 Message-Id: <200803221423.m2MENhMM029693@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: grading guidelines question Date: Sat, 22 Mar 2008 07:23:43 -0700 From: Bill Cheng Someone wrote: > The regular nodes delete their log files when they do a soft restart. Now > after starting again, they don't find any beacon node to connect. And since > there are no connections, there should be no records in the log file. So the > log files are empty... The log file should *not* be deleted when the node restarts! You should only delete it when you start the node with the -reset commandline option. I just updated the the spec to make this explicit. -- Bill Cheng // bill.cheng@usc.edu -----Original Message----- From: Bill Cheng [mailto:william@bourbon.usc.edu] Sent: Friday, March 21, 2008 11:03 PM To: cs551@merlot.usc.edu Subject: Re: grading guidelines question Someone wrote: > This refers to grading guidelines B-2-c > > one beacon node, 3 regular nodes, linear network, killing the only beacon > node > > this test case asks about the log files of regular nodes.. but since the > only beacon node dies, all the regular nodes restart and shut down after not > finding any beacon node. So their log files will not contain anything. Are > these supposed to be empty? The guidelines doesn't specify about it. What about CHECK, NOTIFY, and may be HELLO? -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 22 07:40:40 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2MEeeQD032374 for ; Sat, 22 Mar 2008 07:40:40 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2MEED7P029282 for ; Sat, 22 Mar 2008 07:14:13 -0700 Message-Id: <200803221414.m2MEED7P029282@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: grading guideline question Date: Sat, 22 Mar 2008 07:14:13 -0700 From: Bill Cheng Someone wrote: > I have a question regarding to a grading guideline: > > ======================================================================== > ============================ > (d) (+3 points) a single beacon node with a regular node joining the > network, the join would fail because there is not enough > neighbors > > Restart the beacon node in the first window as follows: > ~csci551b/bin/resetf1 > /bin/rm ??.out > ./sv_node r5-n00.ini > (wait till next node get started, then wait 10 > seconds, > at the commandline prompt, type > "status neighbors 1 00.out", look at "00.out") > > in 2nd window, do: > ./sv_node r5-n05.ini > (this node has InitNeighbors=3, join should fail > after 5 > seconds, this node should give up and die) > ======================================================================== > ============================ > > What should go into "00.out" file? nothing (since node n05 just tried > to JOIN to n00 and later failed - not consider as a connection > establishment) or there should be a topology (2 nodes and a link of a > single beacon node with a regular node)? JOIN requests should be logged. If there are other nodes the beacon node is connected to, it will have to flood the JOIN request. So, it should have a record of where the request came from. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Mar 21 23:29:23 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2M6TNfD020694 for ; Fri, 21 Mar 2008 23:29:23 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2M62wWR020618 for ; Fri, 21 Mar 2008 23:02:58 -0700 Message-Id: <200803220602.m2M62wWR020618@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: grading guidelines question Date: Fri, 21 Mar 2008 23:02:58 -0700 From: Bill Cheng Someone wrote: > This refers to grading guidelines B-2-c > > one beacon node, 3 regular nodes, linear network, killing the only beacon > node > > this test case asks about the log files of regular nodes.. but since the > only beacon node dies, all the regular nodes restart and shut down after not > finding any beacon node. So their log files will not contain anything. Are > these supposed to be empty? The guidelines doesn't specify about it. What about CHECK, NOTIFY, and may be HELLO? -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Mar 21 23:15:24 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2M6FOIB019483 for ; Fri, 21 Mar 2008 23:15:24 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2M5n0TS020173 for ; Fri, 21 Mar 2008 22:49:00 -0700 Message-Id: <200803220549.m2M5n0TS020173@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: 551 FP1 "check msg query" Date: Fri, 21 Mar 2008 22:49:00 -0700 From: Bill Cheng Someone wrote: > Suppose I am a regular node with 4 active links with my neighbors. > > If one of my links breaks, I will send flood a check message. > Before I get a response for my check request (while I am waiting), another > link breaks. > > What should I do? > 1. Should I flood another check request for the second link that broke. > 2. Or should I reset my check timer. You should flood another CHECK. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Mar 21 23:03:41 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2M63f6u018419 for ; Fri, 21 Mar 2008 23:03:41 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2M5bHHK019787 for ; Fri, 21 Mar 2008 22:37:17 -0700 Message-Id: <200803220537.m2M5bHHK019787@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: HLLO in log file Date: Fri, 21 Mar 2008 22:37:17 -0700 From: Bill Cheng Someone wrote: > Grading guidelines A(2) says, > > in 3rd window, do: > more ~/tmp/final1/n00/servant.log > (and see what goes into the log file; > this file should have things like HLLO, KPAV, NTFY, > STRQ, and STRS; there should be 2 HLLOs, one on > sending and one on receiving) > > I have a doubt regarding '2 HLLOs' in the log file. Should it be 2 *pair* of > HLLOs or only *2* HLLOs?? As each beacon node will make 2 connections with > the other one and terminates one connection after exchanging HLLO messages, > there should be 4 HLLO messages in the log file (2 sent and 2 received). > > Or is it only 2 HLLO messages? Since we start the nodes one at a time, most likely, you should only have one connection between the two nodes. Therefore, 2 HELLO messages. It is possible that when you start the 2nd node, the 1st node just happens to be retrying its connection with the 2nd node, then you may end up with 4 HELLO messages. This can happen if you time it just right. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Mar 21 22:57:31 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2M5vUrv017877 for ; Fri, 21 Mar 2008 22:57:30 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2M5V6qJ019559 for ; Fri, 21 Mar 2008 22:31:06 -0700 Message-Id: <200803220531.m2M5V6qJ019559@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: regarding Final Project: beacon connections Date: Fri, 21 Mar 2008 22:31:06 -0700 From: Bill Cheng Someone wrote: > From the Spec understanding , the beacon nodes form a network > that is fully connected, always. > > What is required for the following two scenarios: > > Scenario 1 > A beacon node comes up and tries to form connections with all > other beacon nodes. > In case connection to one of the beacon node could not be made , > then > ---should it start processing other messages like JNRQ , SHRQ, > from other nodes and Re-try in parallel > or > --should it first complete connections with all the beacon nodes > and then only start processing other messages like JNRQ.. Since this is a peer-to-peer network, you don't have control over what nodes are up and what nodes are down. If you wait for all beacon nodes to come up, that may never happen! > Scenario 2 > Consider a fully connected network of beacon nodes. > Now one of these beacon node , say beacon_A, goes down. Then, > -- Should the other beacon nodes stop processing messages from > other nodes and attempt to first re-connect with beacon_A or > -- should they do this in parallel , serving these other > messages like JNRQ , SHRQ, etc or > -- should they send a check message and if they get a response > from any other beacon, they would again start processing all > other requests . The node that went down may never come up. So, you cannot wait for it forever. By the way, a beacon node should never send a CHECK message. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Mar 21 08:33:22 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2LFXMRP005779 for ; Fri, 21 Mar 2008 08:33:22 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2LF73KH004994 for ; Fri, 21 Mar 2008 08:07:03 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m2LF73Xo004993 for cs551@merlot; Fri, 21 Mar 2008 08:07:03 -0700 Date: Fri, 21 Mar 2008 08:07:03 -0700 From: Bill Cheng Message-Id: <200803211507.m2LF73Xo004993@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Copying some stuff about e-mailing question to the Projects web page... Hi everyone, I have copied some stuff about e-mailing me questions from the course description page to the projects web page. Please see: http://merlot.usc.edu/cs551-s08/projects.html#email -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 20 22:42:45 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2L5gjnD017652 for ; Thu, 20 Mar 2008 22:42:45 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2L5GTVf025822 for ; Thu, 20 Mar 2008 22:16:29 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m2L5GTdk025821 for cs551@merlot; Thu, 20 Mar 2008 22:16:29 -0700 Date: Thu, 20 Mar 2008 22:16:29 -0700 From: Bill Cheng Message-Id: <200803210516.m2L5GTdk025821@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Minor changes to the grading guidelines... Hi everyone, I add a few tests to the grading guidelines. Basically, these tests try to see if you update the in-memory list of neighbors properly. Before, in the grading guidelines, there were a few places where I shutdown a list of nodes in sequence. I've added some "status neighbors" commands after I shutdown some of the nodes to check if the topology is correct, before I shutdown the rest of the nodes. The bottomline is that these do not really change anything. Just a few more tests. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 20 22:10:51 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2L5Apmw014779 for ; Thu, 20 Mar 2008 22:10:51 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2L4ia9K025138 for ; Thu, 20 Mar 2008 21:44:36 -0700 Message-Id: <200803210444.m2L4ia9K025138@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 about neighbor list Date: Thu, 20 Mar 2008 21:44:36 -0700 From: Bill Cheng Someone wrote: > How many response of join messages would be enough to create neighbor list? You need to wait till JoinTimeout. Then you count the number of join response messages you have received. If that number is less than InitNeighbors, then you give up. Otherwise, you sort the responses and write the InitNeighbors winners into $HomeDir/init_neighbor_list file and do a soft-reboot. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 20 13:20:35 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2KKKZfM032410 for ; Thu, 20 Mar 2008 13:20:35 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2KJsM3T014835 for ; Thu, 20 Mar 2008 12:54:22 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m2KJsMtl014834 for cs551@merlot; Thu, 20 Mar 2008 12:54:22 -0700 Date: Thu, 20 Mar 2008 12:54:22 -0700 From: Bill Cheng Message-Id: <200803201954.m2KJsMtl014834@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Project group commitment forms... Hi everyon, Just a reminder that if you are doing a group project and have not sent in your group commitment form, please do so before the submission deadline. Once I run MOSS on your submission, it's definitely too late. Also, I must get a commitment form from each partner in order for it to count! Please see: http://merlot.usc.edu/cs551-s08/projects.html#group -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 20 08:25:10 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2KFPAX2005571 for ; Thu, 20 Mar 2008 08:25:10 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2KEwxAG008473 for ; Thu, 20 Mar 2008 07:58:59 -0700 Message-Id: <200803201458.m2KEwxAG008473@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Final project - name of init_neighbor_list file Date: Thu, 20 Mar 2008 07:58:59 -0700 From: Bill Cheng Someone wrote: > As far I understand there is no specification on the name of the > init_neighbor_list file. The spec is very clear about this! It's: init_neighbor_list file in the node's home directory You must use "init_neighbor_list" as the file name! When we do grading of part (2), we will install our init_neighbor_list file in a node's home directory and expect it to work! If you call it something lese, you will be losing some points. > Can we use something like > "init_neighbor_list-port_number" similar to the the > start-port.ini file to distinguish between nodes working in same > directory? Every node has a separate home directory so there should be no conflict. If that's not the case, you are doing something wrong. Please see examples in the grading guidelines. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 19 22:35:48 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2K5ZmvN017152 for ; Wed, 19 Mar 2008 22:35:48 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2K59e9j028461 for ; Wed, 19 Mar 2008 22:09:40 -0700 Message-Id: <200803200509.m2K59e9j028461@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt regarding Fp1 Date: Wed, 19 Mar 2008 22:09:40 -0700 From: Bill Cheng Someone wrote: > consider the following scenario : > Node A is a beacon node > Node B is beacon node. > > Node A has port number greater than Node B. > my algorithm says that a beacon node should try to connect to all other > beacon nodes which are higher than self port number , by this way i am > trying to avoid loop(multiple connection). > > Now if A comes "Up" after B, and B had already tried connect to A > (according to algorithm ). For first attempt B couldn't connect to A ,then > it sleeps for Retry seconds and again tries connect which is successful as A > is up now. > > So is it violating spec for immediate connection of beacons to beacons ? > I thought this as little compromise on performance although functionality > remains same. Well, I wouldn't do it this way. You need to handle the situation when two nodes connect to each other simultaneously for non-beacon nodes anyway. I would just use the code for beacon nodes. If you use the above logic for non-beacon nodes, it may not work. I've discussed this in class. If you are doing this for beacon-only networks, I think it should work. But if you only worry about beacon-only networks, you are not implementing to the spec anyway! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 19 21:54:04 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2K4s4vD013517 for ; Wed, 19 Mar 2008 21:54:04 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2K4Ru1T027905 for ; Wed, 19 Mar 2008 21:27:56 -0700 Message-Id: <200803200427.m2K4Ru1T027905@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: Notify doubt Date: Wed, 19 Mar 2008 21:27:56 -0700 From: Bill Cheng Someone wrote: > If a node receives a 'NOTIFY' message from its neighbor, should it verify > that remaining count of neighbors is atleast equal to 'MinNeighbors' and > make a soft restart if this condition is not satisfied? OR it should not > worry about 'MinNeighbors' field once the node is operationl. It should not worry about MinNeighbors once the node is operational. It should simply flood a CHECK message to the remaining neighbors. The definition of MinNeighbors is: The minimum number of neighbors a node needs to establish connections with *when it comes up*. So, the only time you need the value of MinNeighbors is when a non-beacon node *comes up*. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 19 21:50:11 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2K4oAIT013166 for ; Wed, 19 Mar 2008 21:50:10 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2K4O3tk027845 for ; Wed, 19 Mar 2008 21:24:03 -0700 Message-Id: <200803200424.m2K4O3tk027845@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Check message time out Date: Wed, 19 Mar 2008 21:24:03 -0700 From: Bill Cheng Someone wrote: > What is the time out value for Check message? How long > non-beacon node has to wait for a check response.? You can use the same logic as what I mentioned in my message with timestamp "Mon 17 Mar 16:02". -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 19 21:15:33 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2K4FXIF010035 for ; Wed, 19 Mar 2008 21:15:33 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2K3nQnD027238 for ; Wed, 19 Mar 2008 20:49:26 -0700 Message-Id: <200803200349.m2K3nQnD027238@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt regarding Fp1 Date: Wed, 19 Mar 2008 20:49:25 -0700 From: Bill Cheng Someone wrote: > had a question regarding Message time out. > > Suppose a message comes to a node , when should i start the timer(MSG > Timeout) for this message? Yes. As I've mentioned in class a couple of times, you should not literally *start a timer* because you may end up with too many timers. You should use the approach I mentioned in class. > When inserted into the Queue or When message is processed ? There shouldn't be any difference. > Reason am asking is , IF Queue is big then message timer might influence the > behavaiour You should make sure that this doesn't happen! You should pass off transfering a long file to another thread! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 19 15:19:38 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2JMJKuj010828; Wed, 19 Mar 2008 15:19:20 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2JLrEqX020907; Wed, 19 Mar 2008 14:53:14 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m2JLrEjM020906; Wed, 19 Mar 2008 14:53:14 -0700 Date: Wed, 19 Mar 2008 14:53:14 -0700 From: Bill Cheng Message-Id: <200803192153.m2JLrEjM020906@bourbon.usc.edu> To: cs530@merlot.usc.edu, cs551@merlot.usc.edu, cs558@merlot.usc.edu Subject: Being on campus tomorrow... Hi everyone, As it turns out, I will only be on campus between 11:30am and 1:30pm. If you plan to see me, please send me an e-mail to make an appointment during that period. Thanks! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 19 15:12:59 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2JMCxRX010302 for ; Wed, 19 Mar 2008 15:12:59 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2JLkrbd020749 for ; Wed, 19 Mar 2008 14:46:53 -0700 Message-Id: <200803192146.m2JLkrbd020749@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: shutdown question Date: Wed, 19 Mar 2008 14:46:53 -0700 From: Bill Cheng Someone wrote: > If a non beacon node receives a NOTIFY message from its ONLY neighbor, what > should it do: > > 1. As there is no other neighbor, simply make a soft restart and rejoin > the network, no need of sending CHECK message and waiting for the CHECK > RESPONSE -OR- > > 2. Even though there are no more neighbors, try to flood a CHECK > message and wait for timeout, then make a soft restart and rejoin the > network. > > Which option should I implement? (1) and (2) are equivalent since this node has no neighbors! I guess there is a small difference since you can wait for a timeout. But that's kind of silly since you would not have flooded anything CHECK messages. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 17 19:39:52 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2I2dqVF004601 for ; Mon, 17 Mar 2008 19:39:52 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2I2E2eT009243 for ; Mon, 17 Mar 2008 19:14:02 -0700 Message-Id: <200803180214.m2I2E2eT009243@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Office hours tomorrow? Date: Mon, 17 Mar 2008 19:14:02 -0700 From: Bill Cheng Someone wrote: > Will you holding office hours tomorrow. Since it's Spring Break, there are no office hours! I should be in my office between 1pm and 4pm tomorrow. If you want to make sure that I'm there, please send me e-mail to make an appointment for a specific time. Thanks! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 17 16:28:15 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2HNSE4q019616 for ; Mon, 17 Mar 2008 16:28:14 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2HN2PmW006577 for ; Mon, 17 Mar 2008 16:02:25 -0700 Message-Id: <200803172302.m2HN2PmW006577@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt regarding Fp1 Date: Mon, 17 Mar 2008 16:02:25 -0700 From: Bill Cheng Someone wrote: > I have doubt regarding how status command works. > > We have to flood status message until TTL is 0. > > Now , sending node has to wait for all the status responses to each of nodes > received?. How will the sender node know , how long to wait for status > messages or till how many status messsages it has to receive? You can use a small multiple of MsgLifetime to time this out. The assumption is the after MsgLifetime, nodes in the network will no longer able to forward responses back to you, so it's reasonable to timeout. You can use 1.2 as the multiple (I think that's the value I mentioned in class). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 16 19:43:42 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2H2hffe005708 for ; Sun, 16 Mar 2008 19:43:42 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2H2HxB0017371 for ; Sun, 16 Mar 2008 19:17:59 -0700 Message-Id: <200803170217.m2H2HxB0017371@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 fp1: Problem with setupf1 script Date: Sun, 16 Mar 2008 19:17:59 -0700 From: Bill Cheng Someone wrote: > When i got to cd ~/grading/guidelines and run ~csci551b/bin/setupf1 $HOME, > I get the message 'There is no need to run this script from this account!'. > > When I opened setupf1, I found out that it does the following comparison > > $userid=`/usr/ucb/whoami`; > if ("$userid" == "csci551b") { > print "There is no need to run this script from this account!\n"; > exit 0; > } > > So it's somehow equating my username with "csci551b". > > I did some searching on the net and found out that perl uses 'eq' for string > comparison. So i think it should be > if ("$userid" eq "csci551b") > > I think this may be the problem. But i don't know perl so I am not sure. > Can you please check out the problem? The main problem was that there is a "\n" for $userid because I forgot to call chop(). I've also switch "==" to "eq". Please try it again and see if it works. Thanks for catching the bug! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 16 19:33:13 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2H2XDMC004833 for ; Sun, 16 Mar 2008 19:33:13 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2H27Vbx017152 for ; Sun, 16 Mar 2008 19:07:31 -0700 Message-Id: <200803170207.m2H27Vbx017152@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Regarding beacon retry parameter Date: Sun, 16 Mar 2008 19:07:31 -0700 From: Bill Cheng Someone wrote: > I had a doubt regarding Retry parameter in ini file. Suppose i have a > list of beacons , for a node. So i try to connect to those beacons, if > connect is not possible then i sleep for time specified in Retry filed, Or > the implementation should be .that in first attempt i try to connect to as > much beacons as possible and if that count is not equal to the number of > total beacons, then i sleep for "Retry" time, and then start second round of > connection attempt. > Please let me know if details are required. I'm not sure if you are asking about a beacon node or a non-beacon node. For a non-beacon node, it just needs to connect to one beacon in order to do a JOIN. So, it no beacon node is up, it should just give up, print a meaningful message to the user, and quit. There is no need to retry. For a beacon node, it must connect to all the other beacon nodes. If a beacon node is not up, it should keep retrying after the retry timer has expired. It doesn't matter how you implement it. You just need to make sure that you keep retrying. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 16 09:26:55 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2GGQs50014318 for ; Sun, 16 Mar 2008 09:26:54 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2GG1GuC008384 for ; Sun, 16 Mar 2008 09:01:16 -0700 Message-Id: <200803161601.m2GG1GuC008384@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt on final project part1 Date: Sun, 16 Mar 2008 09:01:16 -0700 From: Bill Cheng Someone wrote: > I wanted to clarify regarding the printing of the time in the > log file. It is given that the format for time is in second and millisecond. > > Is it that we have to just do a gettimeofday and print the second and > millisecond > or should we print it like we did in the warm up2(i.e., taking the > difference from the > absolute and log the time) Please see my message with timestamp "Wed 20 Feb 12:41". gettimeofday() returns time from the 1970 epoch. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 16 09:07:34 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2GG7X5J012551 for ; Sun, 16 Mar 2008 09:07:34 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2GFft8b007407 for ; Sun, 16 Mar 2008 08:41:55 -0700 Message-Id: <200803161541.m2GFft8b007407@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: select() error Date: Sun, 16 Mar 2008 08:41:55 -0700 From: Bill Cheng Someone wrote: > At one place in my program, select() is returning with an error. I caught > this error using perror("select") and the output is "select: Operation not > applicable". > > I looked at the man pages for 'select' but didn't find any description for > this error. > > Please suggest any possible reason for it. I don't know what it means. The only thing I can think of is the usual memory corruption bug that manifested itself in this weird way. Please see: http://merlot.usc.edu/cs551-s08/projects.html#segfault -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 15 14:50:24 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2FLoOY1013349 for ; Sat, 15 Mar 2008 14:50:24 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2FLOpoK030627 for ; Sat, 15 Mar 2008 14:24:51 -0700 Message-Id: <200803152124.m2FLOpoK030627@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: Shut down interrupting communication Date: Sat, 15 Mar 2008 14:24:51 -0700 From: Bill Cheng Someone wrote: > When the node is shutdown, if a connection is in progress, sending a > Notify will not be interpreted correctly. I know this is a part 2 > question, but I'd like to design my subsystems to accommodate both > parts. Here's an example: > > Suppose a user stores a file into the network. While it's flooding but > before the file has finished uploading, the user calls shutdown. > Should we simply stop sending data or let the transfer complete or > should we send Notify? The receiver will not know that the next > message is a Notify as it will assume it's part of the file being > uploaded. What should be done in this situation? Since we are doing TCP and have a stream abstraction and we only have a single channel between two nodes, I think we don't have too many choices when a possibly very very long message is followed by another: 1) we are only allowed to stop at message boundaries 2) we allow a message to terminate before all the bytes are sent in order to begin a new message 3) we can stop anywhere abruptly Choice (1) has the problem that if a message is transfering a huge file, we may have to wait a very very long time before we can stop. This is the case you are talking about. Another problem with this is that what if the is a problem somewhere and you never get to finish the data transfer? Should you wait forever? Choice (2) seems good, but it requires byte-stuffing, which may not be desirable. To get this to work, you can declare the the beginning of a message has a fixed pattern, say 4 consecutive bytes of 0xff. So, whenever you see this pattern in the stream of bytes, it's considered to be the beinning of a new message and the previous message is considered interrupted and should be discarded. But what if your data has 4 consecutive bytes of 0xff? Then you have to modify the data or add an escape character so that the data stream will not have 4 consecutive bytes of 0xff. This is sometimes called byte-sutffing. If you use an escape character, then what would happen if your data stream has this character? You probably have to escape the escape character. The point is that, as you can see, it's a bit complicated. Choice (3) has the problem that you need to handle unexpected termination. But you have to do it anyways since the other side can simply crash! So, it's really not that bad! :-) So, I think we should implement (3). If you are sending a long message (or even a short message), you should just close the connection and don't bother to send NOTIFY! But you still have to shutdown gracefully. This means that you should not just kill the sending thread when it's in the middle of a send() or write() call. You should close the socket as soon as it finishes a send() or write(). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Mar 14 23:54:38 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2F6scEu027180 for ; Fri, 14 Mar 2008 23:54:38 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2F6TBIa012544 for ; Fri, 14 Mar 2008 23:29:11 -0700 Message-Id: <200803150629.m2F6TBIa012544@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Problem with grading guideline Date: Fri, 14 Mar 2008 23:29:11 -0700 From: Bill Cheng Someone wrote: > I seem to be having trouble when trying to run the test cases as > per the grading guidelines. I created the following directories > in my home directory - /bin, /tmp, /tmp/final1, /grading, > /grading/guidelines. > > Then I went to ~/grading/guidelines and ran the script > ~csci551b/bin/setupf1 $HOME. I gave the first port number > assigned to me when prompted. This created all the .ini files > with my port numbers and the respective home directories. Now I > faced the following problems: > > 1> In the grading guidelines before I start execution, I first > run '~csci551b/bin/resetf1' which creates all the home > directories inside ~/tmp/final1/. Then I run '/bin/rm n??.ini > ??.out', which removes any n??.ini and ??.out file in the current > directory where my sv_node executable is present. Then I run 'cp > ~csci551b/public/final1/b1-n??.ini .'. But why are we now copying > an .ini file from ~csci551b/public/final1/. Shouldn't we be > copying the files from '~/grading/guidelines/final1/', which > contain the .ini files configured for each of our account. Hmm... I think you are correct! Thanks for catching it. I've just updated the grading guidelines to copy files from ~/grading/guidelines/final1/. > 2> The home directories do not contain the directory called > 'files' and hence my program gives an error message and quits as > per the specification. Should we create the directory 'files' > inside each home directory? But it would be better if it is > incorporated into the script since for every test case it deletes > the contents of the directory and creates them automatically. Your program should call mkdir() if the directory does not exist. This is actually part (2) stuff! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 13 21:35:33 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2E4ZWmn015126 for ; Thu, 13 Mar 2008 21:35:32 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2E4AEKL015981 for ; Thu, 13 Mar 2008 21:10:14 -0700 Message-Id: <200803140410.m2E4AEKL015981@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 1 Date: Thu, 13 Mar 2008 21:10:14 -0700 From: Bill Cheng Someone wrote: > When a regular node has to shutdown because there was no reply to > the check request message, and lets assume that this node is > connected to another regular node, then it would send the node a > notify message. In this case what would be the error code in the > notify message. Since it's neither "user shutdown" nor "unexpected kill signal received", you should use 0 which is a catch-all. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 13 18:51:45 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2E1pj7R032730 for ; Thu, 13 Mar 2008 18:51:45 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2E1QS8J012462 for ; Thu, 13 Mar 2008 18:26:28 -0700 Message-Id: <200803140126.m2E1QS8J012462@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: doubt Date: Thu, 13 Mar 2008 18:26:28 -0700 From: Bill Cheng Someone wrote: > join response: > host port - "tcp port of listening host".should it be the node's port no > that is replying or the one that initiated the join message You need to think about what the join response is for and you would know the answer! (Hint: join resonse is there so that the joining node can figure out what to write into $HomeDir/init_neighbor_list file.) > check response: > uoid - "it should match the corresponding check message header".should we > directly copy the uoid from the check message uoid field? Yes. You need to think about how a message gets back to the requester. If you do not have the UOID if the request message, how can you route a response message back?! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 13 18:39:08 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2E1d8Nm031684 for ; Thu, 13 Mar 2008 18:39:08 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2E1Dpvh012319 for ; Thu, 13 Mar 2008 18:13:51 -0700 Message-Id: <200803140113.m2E1Dpvh012319@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: doubt Date: Thu, 13 Mar 2008 18:13:51 -0700 From: Bill Cheng Someone wrote: > in the status message it says the port no of the replying host. > so 1 node gets a status message then it will reply with its own portno and > hostname . am i right? Please read the spec carefully. I think you are not being complete. Here's what the spec says: Bytes 49-50: TCP port number of the replying host. Bytes 51+: The hostname (not IP address) of the replying host. If the status request type is neighbors, each Data field contains: Bytes (?)+: TCP port number of a neighboring host. Bytes (?)+: The hostname (not IP address) of a neighboring host. Is there something not clear about the spec? > also the distance field in the join response message just gives the location > value of that node. Again, the spec says: Bytes 47-50: Distance to the node requesting to join. So, you need to compute the *distance*. Not the location of the node. During lectures, I might have said that in a join response, the responding node sent back its location. The spec is correct. I remembered incorrectly. Sorry about it! -- Bill Cheng // bill.cheng@usc.edu On 3/13/08, Bill Cheng wrote: > > Someone wrote: > > > regarding all the message formats: > > > > in all the message formats..... wherever host port and hostname are > > mentioned..........are they the hostport and hostname of the sending > node > > always..............? > > Not always. Please read the spec for details. For example, > in a status response message, a node need to say who it's > connected to. So, its neighbor's host port and hostname > would be in such a message. > -- > Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 13 11:45:39 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2DIjdNT027255 for ; Thu, 13 Mar 2008 11:45:39 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2DIKOo1003826 for ; Thu, 13 Mar 2008 11:20:24 -0700 Message-Id: <200803131820.m2DIKOo1003826@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: doubt Date: Thu, 13 Mar 2008 11:20:24 -0700 From: Bill Cheng Someone wrote: > regarding all the message formats: > > in all the message formats..... wherever host port and hostname are > mentioned..........are they the hostport and hostname of the sending node > always..............? Not always. Please read the spec for details. For example, in a status response message, a node need to say who it's connected to. So, its neighbor's host port and hostname would be in such a message. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 13 08:07:35 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2DF7GTW007343; Thu, 13 Mar 2008 08:07:16 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2DEg3AT031877; Thu, 13 Mar 2008 07:42:03 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m2DEg3Ro031876; Thu, 13 Mar 2008 07:42:03 -0700 Date: Thu, 13 Mar 2008 07:42:03 -0700 From: Bill Cheng Message-Id: <200803131442.m2DEg3Ro031876@bourbon.usc.edu> To: cs530@merlot.usc.edu, cs551@merlot.usc.edu, cs558@merlot.usc.edu Subject: Today's office hour delayed by 30 minutes... Hi everyone, Looks like I won't be able to make it in by 10am this morning. So, I'm delaying today's office hour by 30 minutes to start at 10:30am. Sorry about the short notice. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Mar 13 00:16:23 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=AWL,BAYES_20 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2D7GNAp029358 for ; Thu, 13 Mar 2008 00:16:23 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2D6pCl6022640 for ; Wed, 12 Mar 2008 23:51:12 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m2D6pC4P022639 for cs551@merlot; Wed, 12 Mar 2008 23:51:12 -0700 Date: Wed, 12 Mar 2008 23:51:12 -0700 From: Bill Cheng Message-Id: <200803130651.m2D6pC4P022639@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Reminder: final project part (1) is due right after Spring Break! Hi everyone, Final project part (1) is due on 3/24/08 and Spring Break is next week (3/17 through 3/21). Please plan your time well! During Spring Break, I'm planning to be on campus from Monday through Thursday. Most likely, I won't be on campus on Friday. I'll be out of town on the 22nd and 23rd and I'll be at Disneyland on the 24th! So, I won't be able to help much after next Friday. I often gets *all kinds* of requests for extensions. Please note that my policy is that if there is an *extended* outage on nunki (or on the main filesystems that nunki mounts) for over 2 consecutive hours *and* within 48 hours of the submission deadline, I will extend the deadline by one day. Please also remember that my other general policy is, "whatever I offer to you, I must offer it to the whole class!" I will not give individual extensions unless you have a *documented* illness or a *documented* family emergency. I will not give you an extension if your home ISP is out, if you wipe out something on nunki and you cannot log back in, or if you lost some files and cannot recover them in time, and so on. So, please *leave room for surprises* when you plan your time! If there is an extended outage on nunki within 48 hours of the submission deadline, please send me an e-mail. Thanks! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 12 20:50:25 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2D3oPHJ009308 for ; Wed, 12 Mar 2008 20:50:25 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2D3PF5h019470 for ; Wed, 12 Mar 2008 20:25:15 -0700 Message-Id: <200803130325.m2D3PF5h019470@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Proj Doubt Date: Wed, 12 Mar 2008 20:25:15 -0700 From: Bill Cheng Someone wrote: > While testing, will the port numbers be tested for the assigned port numbers > or will it be tested on the grader's port numbers ?? The grader's port numbers. Although you should use only your port numbers when you run your code, your code should *not* check the values of the port numbers! > Also will the .ini > files be modified to the grader's port numbers ? Of course! Please see the grading guidelines! When you use the files from the grading guidelines, please make sure that you have run the scripts at the bottom of: http://merlot.usc.edu/cs551-s08/projects/testcases/p1.html This will change the port numbers the grader would use to your port numbers so you can test with the files in the grading guidelines. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 12 13:23:39 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2CKNdRB002079 for ; Wed, 12 Mar 2008 13:23:39 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2CJwVcJ011272 for ; Wed, 12 Mar 2008 12:58:31 -0700 Message-Id: <200803121958.m2CJwVcJ011272@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 1 Date: Wed, 12 Mar 2008 12:58:31 -0700 From: Bill Cheng Someone wrote: > When a regular node comes up, its goes down the list of beacon > nodes and connects to the first beacon it finds alive and sends > it a join request message. Once it has got all the join > responses, it breaks down this temporary connection to the beacon > and establishes a connection with each of the neighbors it has > decided upon. So should the regular node send a notify message > when closing this temporary connection. It is logically the > correct thing to do, since it actually closing a connection it > opened with a beacon node. Since this is a "special" (and temporary) connection, there is no need to send NOTIFY. Once you have exchanged HELLO on a link, then you should send NOTIFY (or GOOD-BYE) when you disconnect. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Mar 11 07:55:29 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2BEtSZn004744 for ; Tue, 11 Mar 2008 07:55:28 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2BEUVBj010938 for ; Tue, 11 Mar 2008 07:30:31 -0700 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m2BEUVWB010937 for cs551@merlot; Tue, 11 Mar 2008 07:30:31 -0700 Date: Tue, 11 Mar 2008 07:30:31 -0700 From: Bill Cheng Message-Id: <200803111430.m2BEUVWB010937@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Reminder about midterm... Hi, Just a reminder that today's midterm exam will be in MHP 106. Please remember to bring your photo ID. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 10 16:03:30 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2AN3Uup014822 for ; Mon, 10 Mar 2008 16:03:30 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2AMccHQ026990 for ; Mon, 10 Mar 2008 15:38:38 -0700 Message-Id: <200803102238.m2AMccHQ026990@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Question regarding midterm exam Date: Mon, 10 Mar 2008 15:38:38 -0700 From: Bill Cheng Someone wrote: > In multi-homing slide 7, Can customer use local-pref for Customer-> ISP > traffic? If not why? Also can we use this solution instead of local-pref? I think it will also work. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 10 15:50:21 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2AMoLL2013739 for ; Mon, 10 Mar 2008 15:50:21 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2AMPT4r026791 for ; Mon, 10 Mar 2008 15:25:29 -0700 Message-Id: <200803102225.m2AMPT4r026791@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 question regarding midterm exam Date: Mon, 10 Mar 2008 15:25:29 -0700 From: Bill Cheng Someone wrote: > I have following question regarding midterm exam. > > 1.In multi-homing slide 8: I'm not understand what does statement > "Customer learns full BGP routes and load-shares" means?. > Don't other cases customer doesn't learns full BGP?. After Customer has learned all the routes discovered using BGP, the Customer can split the prefixes so that it uses the two links to ISP approximately equally. > 2.In external vs. internal BGP slide 14: I can't figure out why > 138.39.1.0/30 n/w will become unreachable since all routers still active. > There should only be some problems (i.e. the one that advertised to > R2 won't be able to go through R3 and also in opposite way) but > I can't see why it will be unreachable. I just replied to this since someone else had exactly the same question. Please see my message with timestamp "Mon 10 Mar 15:08". > Also in slide 15 I don't understand why the router need another > virtual IP address. Can't it just use the same IP address even its link > goes down such in slide 14? With this configuration, the two virtual interfaces are on *different* networks. 138.39.128.1 is on the 138.39.128.0/30 network and 138.39.128.5 is on the 138.39.128.4/30 network. On slide 15, these two virtuallly interface are not directly connected. IGP routing will have them connected via the physical link shown. But, when that physical link goes down, there are other routes that can connect these two interfaces, so the TCP connection would not break. > In slide 19, Can you clarify or give me an example of statement > "next-hop attribute traverses sub-AS boundaries (assumes single IGP > running - everyone has same > route to next-hop" I can't see how it gonna differ from regular BGP. I think if you read more slides, you will see an example. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 10 15:38:00 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2AMc0I5012405 for ; Mon, 10 Mar 2008 15:38:00 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2AMD8In026489 for ; Mon, 10 Mar 2008 15:13:08 -0700 Message-Id: <200803102213.m2AMD8In026489@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Doubt in BGP Date: Mon, 10 Mar 2008 15:13:08 -0700 From: Bill Cheng Someone wrote: > I had a question on the scaling of the I-BGP mesh using the > confederation of AS method. When we divide the AS into sub ASes, > shouldn't there be one direct connection between each sub AS. No. It's a two-level hierarchy. One sub-AS is special. > In the slide 19 of the E-BGP vs I-BGP slide set, the peripheral > sub-ASes have been connected to the central sub AS, but shouldn't > there be one connection between each peripheral sub AS too? Then it wouldn't be a hierarchy. > So now instead of having a mesh connection between all the BGP > routers, we divide the routers into groups and now there should > be a mesh connection between each group. Isn't this how it is > supposed to be? I don't know why you would say that! The main thing with AS Confederation is to create a two-level hierarchy to scale the I-BGP "mesh" (by destroying the mesh). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 10 15:33:01 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2AMX1Ge012043 for ; Mon, 10 Mar 2008 15:33:01 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2AM8A7R026375 for ; Mon, 10 Mar 2008 15:08:10 -0700 Message-Id: <200803102208.m2AM8A7R026375@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Midterm doubt Date: Mon, 10 Mar 2008 15:08:10 -0700 From: Bill Cheng Someone wrote: > I have a question about "Failure on I-BGP Link" (slide 14 External v.s. > Internal BGP). > Why the network 138.39.1.0/30 becomes unreachable when R1-R2 goes down. >From the picture, it looks like when the physical link between R1 and R2 breaks, 138.39.1.1 can still reach 138.39.1.2 via R3. I think what happens is that routing cannot have loops, there is only one way to go from R3 to the network between R1 and R2. Therefore, when the link in question breaks, one of the router interface cannot reach the other router interface in question. (This would not be true if virtual interfaces are used.) -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 10 10:02:27 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2AH2RCI014260 for ; Mon, 10 Mar 2008 10:02:27 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2AGbbEa017778 for ; Mon, 10 Mar 2008 09:37:37 -0700 Message-Id: <200803101637.m2AGbbEa017778@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Midterm doubt Date: Mon, 10 Mar 2008 09:37:37 -0700 From: Bill Cheng Someone wrote: > According to Inter-domain Routing sheet: slide 42 > A prefix after aggregation shown is 10/8:e(3 [1 2]). However, why > should we aggregate these prefix into 10/14? since > 10.00000001 > 10.00000010 > 10.00000011 > as can be seen, the longest prefix that we can aggregate is 14. > so the answer would be 10/14:e(3 [1 2]). In that example, there are more networks in AS3 that are not drawn. When you look at a picture, you need to decide if there are things that are not drawn. During exams, if you are not clear about a particular question, you should come ask me to clarify. By the way, if you write addresses like: 10.00000001 10.00000010 10.00000011 where you mix decimal and binary digits; although I know what you mean, you may not receive full credit because this is not a valid notation. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 10 08:16:39 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2AFGd1R004838 for ; Mon, 10 Mar 2008 08:16:39 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2AEpoWE014618 for ; Mon, 10 Mar 2008 07:51:50 -0700 Message-Id: <200803101451.m2AEpoWE014618@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: 'reserved' field Date: Mon, 10 Mar 2008 07:51:50 -0700 From: Bill Cheng Someone wrote: > We have a 'reserved' field in the header of all the messages in Final > project, which is always set to zero. I found something relevant to this in > RFC 2795, a jocular RFC, published on April fool's day... > > ".every other protocol specification includes a "future use" reserved field > which never, ever changes and is therefore a waste of bandwidth and memory." The statement it true by definition! That's why it's kind of a silly statement! If it defines what "never" means, then it would be more interesting. There is another saying, "never say never"! :-) -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 10 08:12:30 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m2AFCUns004458 for ; Mon, 10 Mar 2008 08:12:30 -0700 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2AElepV014516 for ; Mon, 10 Mar 2008 07:47:40 -0700 Message-Id: <200803101447.m2AElepV014516@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Midterm Scope Doubt Date: Mon, 10 Mar 2008 07:47:40 -0700 From: Bill Cheng Someone wrote: > According to the slide : Routing Review, I remembered that there are > some topic that we have not been went through in class; > - Sequence Number Space Issues (Slide no 54) > - One Solution: JSP Aging (Slide no. 55) > - A better Solution (Slide no. 56) > - Lollipop Operation (Slide no. 57) > - Lollipop Operation (Cont...) (slide no. 58) > - Is Aging still Need? (Slide no. 59) > Will these topics be covered in the midterm? Yes. In the News section, the item on 3/5/2008 says explicitly that: Here is a quick summary of the topics (not all topics covered are listed): ... So, it's intentional that *not all* topics covered are listed. Therefore, you should fall back to what's said in the above paragraph: The midterm exam will cover everything from the beginning of the semester ... Please read the "..." from the web page to see exactly what's covered. With regard to the specific slide numbers you mentioned above, these were review from EE 450. Although they were skipped during lectures, they are till part of the midterm coverage. As I mentioned before, most questions on the midterm will come from lectures, although you are responsible for all the required materials (except the ones explicitly excluded); but I reserve the rights to ask questions from the required materials that were not covered during lectures. (This sounds like a legal document! But it's what I mentioned in class, and I guess it's necessary to repeat it here.) -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 8 20:19:58 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m294JwZi011454 for ; Sat, 8 Mar 2008 20:19:58 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m293tKK8019610 for ; Sat, 8 Mar 2008 19:55:20 -0800 Message-Id: <200803090355.m293tKK8019610@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt on the midterm Date: Sat, 08 Mar 2008 19:55:20 -0800 From: Bill Cheng Someone wrote: > In the homework problem on landmark routing, it is asked to write > an algorithms for landmark routing. So is this the algorithm of > how one should select the nodes for each level. Also does this > algorithm need to give optimal solution for example, In the "extra homework", probme 1 is not asking for an optimal solution, but problem 2 is asking about an optimal solution because it uses the word "minimum". > in the home > work problem with only one LM1 router i can reach all the nodes > in that network in four hops, but the solution says 3. Hence can > i assume that my algorithm only needs to find the set of Level i > router, but on the optimal set. But it would not satisfy the "red" rule (please see slide 17 of lecture 8). Please remember that the key phrase in the red rule is "is seen" and *not* "sees". I think you got it backwards and that's they you thought that you only need 1 LM1 router. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 8 20:12:59 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m294CxTD010911 for ; Sat, 8 Mar 2008 20:12:59 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m293mMmA019500 for ; Sat, 8 Mar 2008 19:48:22 -0800 Message-Id: <200803090348.m293mMmA019500@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt on the midterm Date: Sat, 08 Mar 2008 19:48:22 -0800 From: Bill Cheng Someone wrote: > I have a doubt regarding the BGP policies. For example I have two > AS's AS1(11/8) and AS2(12/8) connected by two links,(as on slide > 43 of feb 21st lecture). > > AS1 exports to AS2 > 11/8:a(1) > 11/8:c(1 1) > > But AS2 as already set the local pref in its routing table > 11/8:b(1)w/LP=5 > 11/8:d(1)w/LP=10 > > So will the traffic go on link a-b or c-d from AS2 to AS1 On slide 55 of lecture 11 (Feb 21), it says that LOCAL_PREF has preference over AS_PATH length. So, for your example, the AS_PATH length will not even be looked at! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Mar 7 23:19:01 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m287J1Md027725 for ; Fri, 7 Mar 2008 23:19:01 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m286sVgi026491 for ; Fri, 7 Mar 2008 22:54:31 -0800 Message-Id: <200803080654.m286sVgi026491@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: nam error Date: Fri, 07 Mar 2008 22:54:31 -0800 From: Bill Cheng Someone wrote: > As the flooding algorithm produces the 'nam' file about the network, > sometimes I get info about a link before I have info about the connected > node. For example, > > (an actual output of my program) > > V -t * -v 1.0a5 > n -t * -s 12313 -c red -i black > n -t * -s 12312 -c red -i black > l -t * -s 12312 -d 12311 -c blue > l -t * -s 12312 -d 12313 -c blue > n -t * -s 12311 -c red -i black > l -t * -s 12311 -d 12313 -c blue > l -t * -s 12311 -d 12312 -c blue > > Here, the fourth row describes a link between 12312 and 12311 but the > information about node 12311 is in row 6. So the "nam file.out" command is > throwing error. What should I do to ensure the order of nodes and links? Please see my message with timestamp "Wed 05 Mar 12:18". So, before you print your 4th row, you should detect that you have not print a node out. So, you print the node first before you print the link. When you print a node, you need to add the node to the list of nodes you've seen before. This way, you can print it on-the-fly. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Mar 7 23:12:58 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m287Cw1I027203 for ; Fri, 7 Mar 2008 23:12:58 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m286mS4x026380 for ; Fri, 7 Mar 2008 22:48:28 -0800 Message-Id: <200803080648.m286mS4x026380@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Doubt on the midterm Date: Fri, 07 Mar 2008 22:48:28 -0800 From: Bill Cheng Someone wrote: > I would like to know if we have to know the > BGP messages format completly with diagram or, is it enough to > know its functionality. You do not need to memorize BGP message format. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 5 22:56:17 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m266uHac024419 for ; Wed, 5 Mar 2008 22:56:17 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m266W3n9000471 for ; Wed, 5 Mar 2008 22:32:03 -0800 Message-Id: <200803060632.m266W3n9000471@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 1 Date: Wed, 05 Mar 2008 22:32:03 -0800 From: Bill Cheng Someone wrote: | On Mar 5, 2008, at 22:00 , Bill Cheng wrote: | > | > I guess this is the difference between UNIX and PC/Mac. The | > UNIX convention is to use the current working directory for | > output. On the PC and Mac, by default, things go to the a | > particular place, i.e., the Desktop, since there is really | > no notion of the current working directory. | | That is not true. Macs are FreeBSD-based and applications can be run | in the console. Just as in Unix, an application can call chdir which | effectively changes the current working directory (man 2 chdir). | Windows has a similar function: _chdir. Applications may also be run | in a pseudo terminal: cmd. Yes, the default location may perhaps be | the Desktop, however, the current working directory is always present. You are right. But I think a lot of people do not use the commandline interface on PCs and Macs. Before cygwin came along, I hated to use the commandline interface on Windows because I could never get used to the backslashes! I'm not even sure if there was a commandline interface on the Mac before Mac OS X for a regular user. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 5 22:39:45 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m266djXh022743 for ; Wed, 5 Mar 2008 22:39:45 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m266FV6s032664 for ; Wed, 5 Mar 2008 22:15:31 -0800 Message-Id: <200803060615.m266FV6s032664@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: new beacon Date: Wed, 05 Mar 2008 22:15:31 -0800 From: Bill Cheng Someone wrote: > After all the beacon nodes have formed a mesh connection, is it possible for > any other node to appear as a beacon node in the SERVANT? I mean a node > which is not there in the [beacons] sections of initial beacon nodes. No. You may assume that the startup configuration files for all the nodes have the same list of beacons. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 5 22:24:15 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m266OFid021485 for ; Wed, 5 Mar 2008 22:24:15 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m26601Fb032010 for ; Wed, 5 Mar 2008 22:00:01 -0800 Message-Id: <200803060600.m26601Fb032010@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 1 Date: Wed, 05 Mar 2008 22:00:01 -0800 From: Bill Cheng Someone wrote: > The requirements for the status response nam file says " > is the name of an external file (relative to the current working > directory if it does not begin with a / character)." Shouldn't > this file for each node be in the respective home directories > since if in all my windows I give a status command with the same > file name, then all four processes will try to create the same > file in the current working directory and will cause an error. It should go into the current working directory. A node's HomeDir, in a way, belongs to the server side of the node. The client side of the node uses the current working directory. I guess this is the difference between UNIX and PC/Mac. The UNIX convention is to use the current working directory for output. On the PC and Mac, by default, things go to the a particular place, i.e., the Desktop, since there is really no notion of the current working directory. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 5 22:18:17 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m266IGd7020951 for ; Wed, 5 Mar 2008 22:18:16 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m265s3Ps031930 for ; Wed, 5 Mar 2008 21:54:03 -0800 Message-Id: <200803060554.m265s3Ps031930@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Final Project doubt Date: Wed, 05 Mar 2008 21:54:03 -0800 From: Bill Cheng Someone wrote: > I am using the following structure for the common header. But when I do > sizeof for this structure it says 28 bytes and not 27. The last field > datalength is getting aligned and hence is pushed ahead by one byte.How do i > overcome this to have a 27 byte header. > > > typedef struct header{ > uint8_t msg_type; > char UOID[20]; > uint8_t TTL; > uint8_t reserve; > uint32_t data_length; > }COMMON_HEADER; This is what I was illuding to on slides 2 and 3 of lecture 2! I think the reason is that nunki is a RISC machine. RISC machine likes data to be word aligned. So, everything is a multiple of 4 bytes! So, I would recommend to follow slide 4 of lecture 2 and use an array of unsigned characters when you send and receive data. You can copy the fields into the array when you write and copy bytes from the array into the fields when you read. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 5 19:19:21 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m263JKjA004647 for ; Wed, 5 Mar 2008 19:19:20 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m262t8SZ027659 for ; Wed, 5 Mar 2008 18:55:08 -0800 Message-Id: <200803060255.m262t8SZ027659@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: doubt regarding signals Date: Wed, 05 Mar 2008 18:55:08 -0800 From: Bill Cheng Someone wrote: > Do I need to activate the signal handlers in all the threads or only in > main()? > > Means if I activate the signal handler in thread X only, then will the > signal be delivered to all the threads or only to thread X? If you want the signal to go to a particular thread, you not only have to activate it for that thread, you should also block it for all other threads. By default, everything is activated, I think. You should look at the example in "man pthread_sigmask" on nunki to see how to do it right. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 5 13:34:58 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m25LYwi6006033 for ; Wed, 5 Mar 2008 13:34:58 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m25LAm2Y019691 for ; Wed, 5 Mar 2008 13:10:48 -0800 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m25LAmwx019690 for cs551@merlot; Wed, 5 Mar 2008 13:10:48 -0800 Date: Wed, 5 Mar 2008 13:10:48 -0800 From: Bill Cheng Message-Id: <200803052110.m25LAmwx019690@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Midterm exam coverage... Hi everyone, I've mentioned during yesterday's lecture that the midterm exam will cover everything from the beginning of the semester till the end of yesterday's lecture on congestion control (includes materials from part of [Ramakrishnan90a]). I have update the class web page with a summary of the topics. Please see: http://merlot.usc.edu/cs551-s08/#news Anything about "ns" and "nam" are excluded from the exam. I've also noticed that 5 of my slides were misplaced! So, the 5 slides on RTT and RTO (slides 49 through 53 of lecture 13 on 2/28/2008) are also excluded from the midterm exam. The midterm exam will be held on next Tuesday (3/11/08) during class time in MHP 106. It will be closed book, closed notes, and closed everything (and no "cheat sheet"). Also, no calculators, cell phones, or any electronic gadgets are allowed. Please bring a photo ID. Your ID will be collected at the beginning of the exam and will be returned to you when you turn in your exam. There will be assigned seating. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 5 12:42:28 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m25KgSvA001085 for ; Wed, 5 Mar 2008 12:42:28 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m25KIIJD018741 for ; Wed, 5 Mar 2008 12:18:18 -0800 Message-Id: <200803052018.m25KIIJD018741@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: doubt regarding 'status' command Date: Wed, 05 Mar 2008 12:18:18 -0800 From: Bill Cheng Someone wrote: > Will the 'status' command prints the results only after 1.2*MsgLifeTime or > after the user presses CTRL^C? Whichever comes first. Or you can print the results on-the-fly. If you keep a list of node names that you have seen since the start of the command, you can actually figure out what you need to print as soon as you see a reply. It's okay to print a link twice since the "nam" program doesn't mind! I've mentioned this in my message with timestamp "Sun 02 Mar 09:48". > Is it true for all the command that hides the command line prompt untill > resutls are available (like status, get, search)? It's the opposite. You should always try to print the results on-the-fly, if you can. Since we don't have to rank the results, we don't have to wait (in all the cases that I can think of). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Mar 5 12:35:55 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=AWL,BAYES_20 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m25KZtNN000379 for ; Wed, 5 Mar 2008 12:35:55 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m25KBjlf018614 for ; Wed, 5 Mar 2008 12:11:45 -0800 Message-Id: <200803052011.m25KBjlf018614@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Final Project part 1 clarification Date: Wed, 05 Mar 2008 12:11:45 -0800 From: Bill Cheng Someone wrote: > I just wanna clarify several issues regarding the final project: > > (1) How does the program know whether the sv_node is a beacon > node? Is it just by looking at the command line "sv_node > start-12311.ini" and check whether the given port number is in > the beacon section of the ini file? No. From the startup configuration, the node can read the Port key in the [init] section to find out what port it should listen to. It should call gethostname() to get the hostname for the machine it runs on. Then it should form a string "HOSTNAME:PORT" using the obtained values and see if this key is in the [beacons] section of the startup configuration file. > (2) Do we assume the given ini file name is always be > "start-port.ini"? No. The startup configuration file name can be anything. If you look at the grading guidelines, They are not like "start-port.ini". > (3) I am bit confused about the connection initializing process > in beacon and non-beacon nodes. What I get from the spec is that > when a beacon node comes up, it reads the beacon section in the > given ini file and initializes connections to all those beacon > nodes in an order by comparing the port numbers and host names. I'm not sure what you meant by the last line. There is no order and there is really nothing to compare to. I guess if the node is a beacon node, it should not connect to itself. Is that what you meant by comparison? You can contact the other beacon nodes in any order you want. > After each connection established, the two nodes exchange hello > messages and from that point on the connection is deemed to be > operational. Correct. And if there are two connections between a pair of beacon nodes, you need to close out one connection using the algorithm described in the spec. > For a non-beacon node, it needs to go through the > join process first and then choose which nodes to connect. After > it initialized connection to the end node and they finished > exchanging hello messages, the connection is deemed to be > operational. Correct, although I wouldn't put it this way. A non-beacon node can start and stop by the user at any time. When it gets started, it should check if the init_neighbor_list file exists in its HomeDir. If it is, there is no need to join. If it is not, then it needs to join. > Does it mean > only beacon nodes need to compare port numbers and host names to > determine the connection order to other beacon nodes? (Sorry, > it's a bit verbose) I think I understand what you meant by comparisons now. You should not do the comparison when you decide which node to connect to. If you do this, it would be possible that the beacon nodes are not connected in a full mesh. I explained this possibility during class. So, a beacon node should connect to all other beacon nodes. The comparison you talked about is regarding "breaking a tie". This needs to be done with *any* pair of nodes. Not just a pair of beacon node. This can happen even if two non-beacon nodes are connecting to each other (when they appear in each other's init_neighbor_list file). After you have exchanged HELLO messages between two nodes, you must check for the tie breaking condition if you have detected that there are two connections between this pair of nodes. > (4) We suppose not to send and recv one byte at a time for this > project, right? Correct. You should not do it. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Mar 4 21:34:19 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m255YJHK013571 for ; Tue, 4 Mar 2008 21:34:19 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m255AEVr031520 for ; Tue, 4 Mar 2008 21:10:14 -0800 Message-Id: <200803050510.m255AEVr031520@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project part 1 Date: Tue, 04 Mar 2008 21:10:14 -0800 From: Bill Cheng Someone wrote: > In the Notify message what is the difference between the error > code 1 and 2. 1 is user shutdown which the user does using > Cntrl+C and 2 is unexpected kill signal received which is also > Cntr+c. The spec clearly states that "^C (Control-c) should never cause your program to terminate." Please make sure you code it that way! > There should be only two error codes. One for normal > shutdown and other for some internal error that caused the thread > to terminate such as a broken pipe or something. Please use 1 for user shutdown, 2 for any kind of kill signals (such as SIGTERM, i.e., kill -9), and 0 for everything else (if there is anything else). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Mar 3 07:54:34 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50,WEIRD_PORT autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m23FsYDw005133 for ; Mon, 3 Mar 2008 07:54:34 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m23FUg96027695 for ; Mon, 3 Mar 2008 07:30:42 -0800 Message-Id: <200803031530.m23FUg96027695@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Proj doubt Date: Mon, 03 Mar 2008 07:30:42 -0800 From: Bill Cheng Someone wrote: > In the beacons section of .ini file::: > foo.usc.edu:12311= > foo.usc.edu:12312= > Is something goin to be there after the equal to sign,if there is > what is it going to be. Nothing! In Windows INI terminology, what's on the left of the equal sign is called a "key" and what's on the right of the equal sign is called a "value" (for the corresponding key). For the above example, these keys have no values and that's perfectly fine. The usual way to handle this is to obtain all keys in a section first. Then for each key, you can get its value (and make sure your code handles null values). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 2 20:30:41 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m234Ufdq008292 for ; Sun, 2 Mar 2008 20:30:41 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2346rga010532 for ; Sun, 2 Mar 2008 20:06:53 -0800 Message-Id: <200803030406.m2346rga010532@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Regarding final project. Date: Sun, 02 Mar 2008 20:06:53 -0800 From: Bill Cheng Someone wrote: > If we see our system as a file system , then it might behave > slightly different from other file systems. Then you should probably not view the SERVANT network as a filesystem. > Since in our case - it might happen that if we execute a delete > file command, the file might still persist.There is no way to > tell whether it has been deleted from the system or not. > > In such a practical , deployed distributed file system - is this > difference in functionality acceptable to the customers ? > > Or is that the target users ,for which this file system is > developed , are fine with this ? I'm not sure what would be an exact definition of a filesystem. But if SERVENT does not fit the definition, it should not be called a filesystem. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 2 20:25:19 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m234PJnM007762 for ; Sun, 2 Mar 2008 20:25:19 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2341VRg010247 for ; Sun, 2 Mar 2008 20:01:31 -0800 Message-Id: <200803030401.m2341VRg010247@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: query Date: Sun, 02 Mar 2008 20:01:31 -0800 From: Bill Cheng Someone wrote: > What is an event for the dispatcher? Slide 19 of leture 10 is "just a design". What's depicted is called an "event driven" design. Here, the word "event" has nothing to do with pthread. An "event" is just a C/C++ structure/object. So, if you want to go this way, you would turn everything into an "event" and pass around "events". > Is it an element of a list which contains the full message? Whatever it takes to get your program to work properly! You get to define what events you have and what's in each type of event. > Also, does each connection has its separate list > and the event dispatcher copies an element from its list to the appropriate > connection's list? It's up to you! You can do it in many different ways and there is probably no right way or wrong way. Whateve that's easier for you to understand and keep things straight. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Mar 2 10:12:01 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m22IC0ig018512 for ; Sun, 2 Mar 2008 10:12:00 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m22HmFTu031148 for ; Sun, 2 Mar 2008 09:48:15 -0800 Message-Id: <200803021748.m22HmFTu031148@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Final Project Date: Sun, 02 Mar 2008 09:48:15 -0800 From: Bill Cheng Someone wrote: > Question regarding nam file. Can we write duplicates > statements in status.out file. for example > > V -t * -v 1.0a5 > n -t * -s 3 -c red -i black > n -t * -s 4 -c red -i black > n -t * -s 5 -c red -i black > n -t * -s 6 -c red -i black > n -t * -s 7 -c red -i black > l -t * -s 7 -d 6 -c blue > l -t * -s 6 -d 7 -c blue # this is a duplicate link > l -t * -s 6 -d 5 -c blue > l -t * -s 5 -d 6 -c blue # this is a duplicate link > l -t * -s 5 -d 4 -c blue > l -t * -s 5 -d 3 -c blue > l -t * -s 4 -d 5 -c blue # this is a duplicate link > > the output comes perfect even with the duplicate link present in the file. Yes! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 1 20:33:40 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m224XeFJ009800 for ; Sat, 1 Mar 2008 20:33:40 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m2249x03026707 for ; Sat, 1 Mar 2008 20:09:59 -0800 Message-Id: <200803020409.m2249x03026707@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Regarding send in final #1 Date: Sat, 01 Mar 2008 20:09:59 -0800 From: Bill Cheng Someone wrote: > While sending data, do we have to send 1 byte at a time ( as > specified in warm up #1 ) ? No. I've mentioned in class that you should not do this. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 1 11:18:14 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=AWL,BAYES_20 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m21JIEAL018447 for ; Sat, 1 Mar 2008 11:18:14 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m21IsaXl014699 for ; Sat, 1 Mar 2008 10:54:36 -0800 Message-Id: <200803011854.m21IsaXl014699@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Regarding message type in final #1 Date: Sat, 01 Mar 2008 10:54:36 -0800 From: Bill Cheng Someone wrote: > Actually i combined two descriptions. It was written in > comm header spec that message type is 1 byte . > and in ""Logging" spec it was mentioned that should > be 4 characters long( which eventually would be 4 bytes), so > i assumed that both are pointing to same thing . MessageType in a message is always one byte long and in the log file is always 4 character long which is the *name* of the corresponding MessageType. For example, if MessageType in a message is 0xFC, you need to log "JNRQ" into the log file. There is no inconsistency here. -- Bill Cheng // bill.cheng@usc.edu ----- Original Message ----- From: Bill Cheng Date: Saturday, March 1, 2008 9:11 am Subject: Re: Regarding message type in final #1 To: cs551@merlot.usc.edu > Someone wrote: > > > Spec says that common header has 1 byte for message type .Also it > > mentions that msgtype is 4 characters long, so if i am not wrong > > , i will have to map these msgtype's to some character or uint_8 > > which is 8 bits long. is this argument ok ? or am i missing > > something from project spec ? > > MessageType is one byte long. I cannot find anywhere in the > spec where it says that msgtype is 4 characters long. Could > you let me know which section it says that it's 4 bytes long > so I can fix it? > -- > Bill Cheng // bill.cheng@usc.edu > Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 1 09:28:38 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m21HScBp006924 for ; Sat, 1 Mar 2008 09:28:38 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m21H51hK011633 for ; Sat, 1 Mar 2008 09:05:01 -0800 Message-Id: <200803011705.m21H51hK011633@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Regarding message type in final #1 Date: Sat, 01 Mar 2008 09:05:01 -0800 From: Bill Cheng Someone wrote: > Spec says that common header has 1 byte for message type .Also it > mentions that msgtype is 4 characters long, so if i am not wrong > , i will have to map these msgtype's to some character or uint_8 > which is 8 bits long. is this argument ok ? or am i missing > something from project spec ? MessageType is one byte long. I cannot find anywhere in the spec where it says that msgtype is 4 characters long. Could you let me know which section it says that it's 4 bytes long so I can fix it? -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Mar 1 09:26:17 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m21HQGsV006697 for ; Sat, 1 Mar 2008 09:26:16 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m21H2dOS011338 for ; Sat, 1 Mar 2008 09:02:39 -0800 Message-Id: <200803011702.m21H2dOS011338@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Regarding final project. Date: Sat, 01 Mar 2008 09:02:39 -0800 From: Bill Cheng Someone wrote: > Considering the design of our peer to peer distributed system, it > might happen that a particular request does not reach every node. Sure! > For e.g. Search request for a file X : > There is a possibility that the file is stored at any node in the > system, but its the search request does not reach this node > because TTL expires. > Same might happen with the Delete command. > > Is this the limitation of our design ? Just like BGP, there is no guarantee that *any* configure you use would be correct and make any network prefix reachable! So, I wouldn't consider it a limitation. It would be a limitation if even if you do everything right, there are situations where a certain objective is not achievable. For example, one of the "limitation" of the SERVANT network is that it does not scale to millions of nodes because we are using flooding. (This is the same limitation of Link State routing.) > If yes, then is there a way to calculate out the TTL value (large > enough) such that the request is fully broad casted? You can always use 255 *if* it is desirable for every request to reach everyone. As you can see from the grading guidelines, it's not alwasy desirable to make that assumption! :-) -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Feb 28 10:14:03 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1SIE3pn007135 for ; Thu, 28 Feb 2008 10:14:03 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1SHogAW014752 for ; Thu, 28 Feb 2008 09:50:42 -0800 Message-Id: <200802281750.m1SHogAW014752@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: doubt Date: Thu, 28 Feb 2008 09:50:42 -0800 From: Bill Cheng Someone wrote: > in the .ini file will it always be mentioned [beacons] or for 1 node it > might be [beacon]? The section name must be [beacons]. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Feb 27 14:49:45 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1RMnjoM025989 for ; Wed, 27 Feb 2008 14:49:45 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1RMQVPB025721 for ; Wed, 27 Feb 2008 14:26:31 -0800 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m1RM9GcP025413 for cs551@merlot; Wed, 27 Feb 2008 14:09:16 -0800 Date: Wed, 27 Feb 2008 14:09:16 -0800 From: Bill Cheng Message-Id: <200802272209.m1RM9GcP025413@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Deleting some test cases for Final Project Part (1)... Hi, I'm deleting test cases t1-1.ini, t1-2.ini, t1-3.ini, t1-4.ini from the following page: http://merlot.usc.edu/cs551-s08/projects/testcases/p1.html One of the problem is that they contain all the optional keys and if you code your startup file parser based on these files, your code may not work with the ones we will test with! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Feb 27 14:39:12 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1RMdC9m025092 for ; Wed, 27 Feb 2008 14:39:12 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1RMFwsd025605 for ; Wed, 27 Feb 2008 14:15:58 -0800 Received: (from william@localhost) by bourbon.usc.edu (8.13.5/8.13.5/Submit) id m1RMFw8q025604 for cs551@merlot; Wed, 27 Feb 2008 14:15:58 -0800 Date: Wed, 27 Feb 2008 14:15:58 -0800 From: Bill Cheng Message-Id: <200802272215.m1RMFw8q025604@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Final project part (1) grading guidelines are available... Hi, I've just posted the grading guidelines for final project part (1). If you see bugs, please let me know. Thanks! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Feb 27 07:54:28 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1RFsSp3020702 for ; Wed, 27 Feb 2008 07:54:28 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1RFVGmQ018231 for ; Wed, 27 Feb 2008 07:31:16 -0800 Message-Id: <200802271531.m1RFVGmQ018231@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 final project Date: Wed, 27 Feb 2008 07:31:16 -0800 From: Bill Cheng Someone wrote: > In the project spec its not stated that for how many join > response message should a regular node wait for to compute the > list of nodes with the minimum distance, this is because some > message might come early and some might take time. You use JoinTimeout to decide when you should stop waiting. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Tue Feb 26 19:56:43 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1R3uhBM019971 for ; Tue, 26 Feb 2008 19:56:43 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1R3XZno005798 for ; Tue, 26 Feb 2008 19:33:35 -0800 Message-Id: <200802270333.m1R3XZno005798@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: cs551 final part 1 Date: Tue, 26 Feb 2008 19:33:35 -0800 From: Bill Cheng Someone wrote: > What should be the interval between the keep alive messages. > Suppose there is no message to transmit then what should be the > time a node should wait before sending a keep alive. The problem > is if this is too short an interval then there is unnecessary > load dumped into the network. Say the interval is 2 seconds but > the keep alive time out is 15 seconds, then if there is message > transmitted every 5 seconds, there will be two keep alive message > transmitted in between which is not needed. Should a node derive > this value from its own keep alive timeout value? If the KeepAliveTimeout is 15 seconds, sending a KeepAlive message every 5 seconds is certainly fine. Or, you can just take half of the KeepAliveTimeout value and take the integer part for your timer. So, in this example, it will be 7 seconds. The actual value is not so important. Sending it every second is certainly an overkill (although it's technically correct). -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Feb 25 19:50:07 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1Q3o7Ed017304 for ; Mon, 25 Feb 2008 19:50:07 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1Q3R7Dh009280 for ; Mon, 25 Feb 2008 19:27:07 -0800 Message-Id: <200802260327.m1Q3R7Dh009280@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP: Logging UOID Date: Mon, 25 Feb 2008 19:27:07 -0800 From: Bill Cheng Someone wrote: > Do we log UOIDs has Hex-formatted strings? According to: http://merlot.usc.edu/cs551-s08/projects/final.html#logging The answer is yes, but you only need to write 4 out of the 20 bytes into the log file in hexstring format. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Mon Feb 25 19:15:40 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1Q3FewR013934 for ; Mon, 25 Feb 2008 19:15:40 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1Q2qfdO008582 for ; Mon, 25 Feb 2008 18:52:41 -0800 Message-Id: <200802260252.m1Q2qfdO008582@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551 Project Date: Mon, 25 Feb 2008 18:52:41 -0800 From: Bill Cheng Someone wrote: > Regarding connection establishment, between nodes A and B. It is > said that ties are broken by comparing port numbers but, who does > the comparision. A recieves port number of B only after > connection establishment. Both sides, after exchanging HELLO messages. They should reach the same conclusion. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Feb 24 23:34:00 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1P7Y0CK005209 for ; Sun, 24 Feb 2008 23:34:00 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1P7B8gA016389 for ; Sun, 24 Feb 2008 23:11:08 -0800 Message-Id: <200802250711.m1P7B8gA016389@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: cs551 final part 1 Date: Sun, 24 Feb 2008 23:11:08 -0800 From: Bill Cheng Someone wrote: > There is a problem I am facing. Suppose only one beacon node > starts up and it tries to connect to the other beacon nodes then > connect fails as expected. The errno value should be returned as > "ECONNREFUSED". I am checking for this value, but when the > connect() fails in my case the errno value is 0. I googled this > up and came up with the cause of the problem. Since we are using > multiple threads in our application we need to give a thread safe > option while compiling and linking in the makefile, so that each > thread uses its own value of errno and not share a global value > of errno. I searched what this option is but I got '-m' which is > for solaris. Could you tell me what this option is for Unix? I just looked at /usr/include/errno.h, it has the following lines: #if (defined(_REENTRANT) || defined(_TS_ERRNO) || \ _POSIX_C_SOURCE - 0 >= 199506L) && !(defined(lint) || defined(__lint)) extern int *___errno(); #define errno (*(___errno())) #else ... #endif /* defined(_REENTRANT) || defined(_TS_ERRNO) */ So, in a multithreaded environment, you don't want errno to be just an integer (for the reason you mentioned above). Therefore, you want errno to be returned by the reentrant function. >From the code in /usr/include/errno.h, it looks like what you need is to define the _POSIX_C_SOURCE flag. This means that when you compile, you need the following compiler flag: -D_POSIX_C_SOURCE I also googled "_POSIX_C_SOURCE pthread errno solaris" and the top pages seem to be useful. Please give this a try and see if it helps. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Feb 24 16:30:54 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1P0UsbI032718 for ; Sun, 24 Feb 2008 16:30:54 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1P084ZG004281 for ; Sun, 24 Feb 2008 16:08:04 -0800 Message-Id: <200802250008.m1P084ZG004281@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Final Project par1: Doubt regarding connections Date: Sun, 24 Feb 2008 16:08:04 -0800 From: Bill Cheng Someone wrote: > I have a small doubt regarding connections priority and breaking > ties. The specifications say that > > "Let PA and PB be the well-known port numbers of nodes A and B, > respectively. If PA > PB, you should keep the connection > initiated by A. If PA < PB, you should keep the connection > initiated by B. If PA == PB, you should compare the hostnames of > nodes A and B." > > Is it possible for two nodes to run using the same well know > port. No. bind() will fail for one of them. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sun Feb 24 16:10:35 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1P0AZlE030787 for ; Sun, 24 Feb 2008 16:10:35 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1ONljHr003852 for ; Sun, 24 Feb 2008 15:47:45 -0800 Message-Id: <200802242347.m1ONljHr003852@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: Status Message Date: Sun, 24 Feb 2008 15:47:45 -0800 From: Bill Cheng Someone wrote: > Is it possible to send a zero-record-length status response message? Yes. But for "status neighbors", this should not happen. If a node does not have a neighbor, it should not have received a status request message. > If so: how do we represent no records? For a status response message, the Data Length in the common header can be used to indicate that there is no Record Length and Data fields. This logic can be used for other message types. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Feb 23 23:14:54 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1O7EsZA005434 for ; Sat, 23 Feb 2008 23:14:54 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1O6qA5X027223 for ; Sat, 23 Feb 2008 22:52:10 -0800 Message-Id: <200802240652.m1O6qA5X027223@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CSCI551 final project Date: Sat, 23 Feb 2008 22:52:10 -0800 From: Bill Cheng Someone wrote: > I have a question regarding connection establishment between two > beacon nodes, say A and B. Suppose node A starts before node B > and its request to node B fails, so A will periodically retry to > establish the connection. Now say node B starts and sends a > connection request from a random port number while it keeps > listening on its well known port number. When node A receives > this request it can know from which port number this request has > been sent, but as this port number is randomly generated port > number node A will no be able to distinguish if this connection > request is from beacon node B or from a regular node, so can the > beacon node use the join message to identify each other when > establishing connections amongst them For any two nodes that are not joining, the first thing they should do is to exchange HELLO messages. In the HELLO message, node B can inform node A what its well-known port number is. Because of this, node A must wait for the HELLO message from the other end of the connection before declaring this connection as a good connection. (Node A should also send a HELLO message to node B immediately on this connection.) By the way, a beacon node *never* sends a join message because it never has to join the network! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Feb 23 23:07:25 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1O77O8v004693 for ; Sat, 23 Feb 2008 23:07:24 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1O6iekj027144 for ; Sat, 23 Feb 2008 22:44:40 -0800 Message-Id: <200802240644.m1O6iekj027144@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: final-1: query Date: Sat, 23 Feb 2008 22:44:40 -0800 From: Bill Cheng Someone wrote: > The spec says, > > "When a new node wishes to join the SERVANT network, it should go down this > list [beacons list] and find a running beacon node to send a join request." > > How does a regular node knows whether a beacon node is running or not just > by checking the ini startup file? Just like warmup project #1, you should try to connect to the well-known port of a node. If you cannot connect, you take it as the node is down and go to the next node on the list. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Feb 23 23:05:10 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=AWL,BAYES_20,WEIRD_PORT autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1O759LQ004511 for ; Sat, 23 Feb 2008 23:05:09 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1O6gPNY027094 for ; Sat, 23 Feb 2008 22:42:25 -0800 Message-Id: <200802240642.m1O6gPNY027094@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: Final Project: Question regarding startup file Date: Sat, 23 Feb 2008 22:42:25 -0800 From: Bill Cheng Someone wrote: > I have a doubt about the format of the configuaration file. > I have written a function (getPrivateProfileString) analogous to > windows API that returns the value corresponding to a key in a > section. For this, I need to have some information in advance > about the structure of the ini file. > > Can I use the following formar for the ini file format: > > [beacons] > beacon1= nunki.usc.edu:14012 > beacon2= nunki.usc.edu:14013 > beacon3= nunki.usc.edu:14014 > beacon4= nunki.usc.edu:14015 > > So that I can make calls to my custom function and get values for > [beacons][beacon1] and so on to get the information about the > beacon nodes in the network. You must *not* change the format of the startup configuration file because we will grade your code with our startup configuration files! All our startup configuration files will *not* have all the lines in the [init] section and the [beacons] section will basically look like the sample given in the spec. For example: [beacons] Retry=15 nunki.usc.edu:12311= nunki.usc.edu:12312= nunki.usc.edu:12313= nunki.usc.edu:12314= So, you really need to be able to handle our startup files. The Windows GetPrivateProfileString() API provides a way to read in all the keys in a section (if the key argument is NULL) and then you can walk through the keys in the section. You should also implement this feature. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Sat Feb 23 22:57:40 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1O6vdOr003799 for ; Sat, 23 Feb 2008 22:57:39 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1O6YtnI027017 for ; Sat, 23 Feb 2008 22:34:55 -0800 Message-Id: <200802240634.m1O6YtnI027017@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP: Packet "limits" Date: Sat, 23 Feb 2008 22:34:55 -0800 From: Bill Cheng Someone wrote: > Do we need to ensure that things such as hostnames are limited to some > arbitrary maximum? Or do we trust the dataLengths set by the sender? You can safely assume that no node is malicious! But you should think about interoperability. A node implemented by another student, if implemented according to the spec, should interoperate with your node! -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Fri Feb 22 22:13:07 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1N6D74X001553 for ; Fri, 22 Feb 2008 22:13:07 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1N5oVLH003280 for ; Fri, 22 Feb 2008 21:50:31 -0800 Message-Id: <200802230550.m1N5oVLH003280@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CSCI551 final project Date: Fri, 22 Feb 2008 21:50:31 -0800 From: Bill Cheng Someone wrote: > For this project can we assume that the ini file is exactly in > the format as give in the example , with no extra spaces or tabs > anywhere. Sorry, but no. You should follow the spec. > for example > Port=12134 will always be this and never Port = 12134 or > Port = 12134 > This is because I want to write my own file parsing code, since > the one give is very complicated and if there is any error then I > wont be able to debug it. If you want to implement your own, please see slide 21 of lecture 10. If you do it that way, you can handle the example you gave above and more. The other way is to use ini_parser mentioned at the bottom of the spec. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Thu Feb 21 08:16:00 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_40 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1LGG0PT024793 for ; Thu, 21 Feb 2008 08:16:00 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1LFrbo6029274 for ; Thu, 21 Feb 2008 07:53:37 -0800 Message-Id: <200802211553.m1LFrbo6029274@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: query regarding make Date: Thu, 21 Feb 2008 07:53:37 -0800 From: Bill Cheng Someone wrote: > Is it ok if I make some subdirectories in my FINAL1 directory and make one > 'makefile' in each subdirectory to generate .o files. Then I will make one > 'makefile' in the FINAL1 directory to generate the 'sv_node' executable. My > idea is that I should divide related files into separate subdirectories > (e.g. 4 files to parse .ini files). OR is it mandatory to put all files in > the same FINAL1 directory? You can organize it any way you want. But you should *not* have a complicated procedure to create "sv_node". Asking the grader to go into each subdirectory to type "make" and then come back to the top level directory to do another "make" is too complicated (you will lose a point or two)! So, you need to write your Makefiles so that everything can be compiled by typing a single "make" at the top level. You can also just use imake or autoconf. Building your executable with only two commands is considered "small variation of make". -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Feb 20 22:10:10 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1L6AAnv002992 for ; Wed, 20 Feb 2008 22:10:10 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1L5lo7q019817 for ; Wed, 20 Feb 2008 21:47:50 -0800 Message-Id: <200802210547.m1L5lo7q019817@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: Connection Preemption Date: Wed, 20 Feb 2008 21:47:50 -0800 From: Bill Cheng Someone wrote: > Can connections be preempted? No. > Example: > Suppose a client (C1) is engaged with a beacon (B1), engaged with > another beacon (B2) in a long, bi-directional file transfer: > ,----, +----+ +----+ > | C1 | <-> | B1 | <-> | B2 | > .----. +----+ +----+ > > Suppose another client, C2 sends some flooding message (such as check): > ,----, +----+ +----+ ,----, > | C1 | <-> | B1 | <-> | B2 | <-- | C2 | > .----. +----+ +----+ .----. > > Do we need to suspend the file transfer from B2 to B1 to service the > flooding request from C2? Or should the flooding request be queued up > at B2 until the file transfer is complete? Since we are using TCP (which is just a stream of bytes), the specification does not have a provision for suspending the transmission of a message. In your example, the CHECK message will not need to be flooded from B2 to B1. Flooding stops when a CHECK message hits a beacon node. > Obviously, this could slow down the network for short check messages, > but we'd need some sort of resume message for lengthy (interruptible) > message types. That's a very good point! I think what you are really saying is that there should be two channels, one for control messages and one for data transfers. Actually, if B2 above is a regular node, because of the topology and the load on the system, starvation can occur. In a random topology and reasonable load on the system, this type of starvation should not happen. Anyway, there is no need to fix this for this project. If this happens, the CHECK message can timeout and C2 will just have to rejoin the network. So, it's really not a problem for the project. -- Bill Cheng // bill.cheng@usc.edu Return-Path: william@bourbon.usc.edu Delivery-Date: Wed Feb 20 13:03:39 2008 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on merlot.usc.edu X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=AWL,BAYES_50 autolearn=no version=3.2.0 Received: from bourbon.usc.edu (bourbon.usc.edu [128.125.9.75]) by merlot.usc.edu (8.14.1/8.14.1) with ESMTP id m1KL3dtc019117 for ; Wed, 20 Feb 2008 13:03:39 -0800 Received: from bourbon.usc.edu (localhost.localdomain [127.0.0.1]) by bourbon.usc.edu (8.13.5/8.13.5) with ESMTP id m1KKfNW0010922 for ; Wed, 20 Feb 2008 12:41:23 -0800 Message-Id: <200802202041.m1KKfNW0010922@bourbon.usc.edu> To: cs551@merlot.usc.edu Subject: Re: CS551: FP1: Time in Logging Date: Wed, 20 Feb 2008 12:41:23 -0800 From: Bill Cheng Someone wrote: > The