WEBVTT 1 00:00:02.010 --> 00:00:09.690 William Cheng: This is the third part of lecture 18 so we just finished rate. And now we're going to take a look at the flash memory system. 2 00:00:10.200 --> 00:00:20.910 William Cheng: So your flash drive. It's like your USB stick. If you have yours. I guess SSD. The, the solid state drive that that's on your laptop. 3 00:00:21.450 --> 00:00:27.180 William Cheng: That's also using the flash technology. Okay, so let's take a look at what's so special about the flash technology. 4 00:00:28.080 --> 00:00:35.880 William Cheng: One good thing about the flash technology is that the flash the way you access the flash device is to access them one block at a time. 5 00:00:36.570 --> 00:00:48.150 William Cheng: So the, you know, so, so there's a really good match between sort of the flash device and the file system blocks, you can sort of map it one for one one block on the flash devices. The same thing as one block inside the file system now. 6 00:00:48.930 --> 00:00:58.290 William Cheng: The other may advantage of the flash devices that it's random access. There's no mechanical parts to it. So there's no seat time. There's no rotational latency. 7 00:00:58.620 --> 00:01:04.740 William Cheng: Accessing the first block or accessing the last blog or access to any blocks on the flash device. The performance is going to be exactly the same. 8 00:01:05.160 --> 00:01:15.660 William Cheng: OK. So again, it's very, very suitable for to be used instead of our system and also will get one of the reason we all like, it is because it's low power. So this way. If you could put any say your laptop. 9 00:01:16.170 --> 00:01:22.380 William Cheng: Your laptop will actually last for long time before you have recharge in the good old days, if you actually have a laptop with the hard drive. 10 00:01:22.890 --> 00:01:30.000 William Cheng: The hard drive, when it spins and there's a motor going on. So, so, so in that case we will we will drain the power 11 00:01:30.690 --> 00:01:37.770 William Cheng: So therefore, using the flash technology, you know, you can keep your laptop running for quite a long time. The last part of his vibration resistant 12 00:01:38.430 --> 00:01:48.390 William Cheng: You know, like my cell phone. I don't know how many times I've dropped it. You know, if I actually have a hard drive in it. The first time I dropped it. It will be the death of my cell phone, right. So now with flash technology. 13 00:01:48.840 --> 00:01:54.420 William Cheng: Its vibration resistance. So I dropped them anytime I everything so. Okay, so what's the, what's the downside of the flash 14 00:01:54.810 --> 00:02:04.170 William Cheng: As it turns out that there's really a strange thing about the flash devices that, as it turns out that your hard drive is actually super reliable, but who mentioned that your hard drive, you probably never seen your hard drive fail. 15 00:02:04.740 --> 00:02:14.340 William Cheng: Bob. Have you ever seen a flash device fail. Okay, just a few years back when I, you know, when I will if I like go to some market and buy those 16 00:02:15.180 --> 00:02:18.930 William Cheng: Little USPS us day, sometimes I use it for two, three weeks that it's dead. 17 00:02:19.380 --> 00:02:25.440 William Cheng: Okay, as it turns out that it was the one kind of file system that will on it. And that's why they die so quickly. Okay, so today we're going to sort of see 18 00:02:25.830 --> 00:02:36.810 William Cheng: What's the problem. And also, what's going to be the solution because the problem with the flash devices that they have limited lifetime compared to this they die pretty quickly. Unless you're you're being very, very careful with it. Okay. 19 00:02:37.350 --> 00:02:46.440 William Cheng: Also there's a symmetric in terms of reading and writing. So reading is really easy to reef on the fly for Dr. Writing. As it turns out, it's more complicated. Okay. In the last part over here, of course, it's 20 00:02:47.250 --> 00:02:59.370 William Cheng: It's more expensive than desk. And again, these numbers are, you know, from a few years back, but, you know, Baba baba still, you know, today the hard drives a lot cheaper to store data compared to the compared to the flash right 21 00:03:01.080 --> 00:03:06.720 William Cheng: Alright, so there are two kinds of flash technology. One is called the norm technology that one's called and then technology. 22 00:03:07.050 --> 00:03:14.310 William Cheng: The norm technology instead of flash device, you can actually read one bite at a time. So typically we don't really use this one instead of file system. 23 00:03:14.490 --> 00:03:23.040 William Cheng: Because instead of houses that we prefer to read a one block at a time. So we're not going to talk about the technology. Yeah, the nano technology. It's page adjustable 24 00:03:23.550 --> 00:03:35.010 William Cheng: Also the, you know, so. So what you do is you take a blog and you divide them into pages a page. As it turns out, you know, it's number that we like it's between one to four kilobytes per page and then 25 00:03:35.430 --> 00:03:45.390 William Cheng: A block of the box. It is about half a megabyte. OK. So again, very, very good match you know for the file system. Right. It's also very, very cheap. So you can just use them to store a lot of data. I mean, again, 26 00:03:46.200 --> 00:03:51.060 William Cheng: It's very cheap relative to memory. But again, it's a lot more expensive than the hard drive. Yeah. 27 00:03:51.870 --> 00:03:59.700 William Cheng: Here's the problem with the the sort of the nanotechnology is that there's a limit on the program erase cycle about 10,000 28 00:04:00.480 --> 00:04:04.020 William Cheng: Guys. So if you look at the flesh device over here. You're going to divide them into blocks now. 29 00:04:04.320 --> 00:04:11.880 William Cheng: As it turns out, you know, the also the one below it over here says that, yeah, there's a different way to to read through through to read the flash device and to right 30 00:04:12.450 --> 00:04:20.670 William Cheng: Right to the flesh device. Will you try to read from the flesh device you can read one page at a time. Okay, well, you try to write to the flesh device, you actually, you have to 31 00:04:21.090 --> 00:04:25.680 William Cheng: Well basically what you have to do is that you have to erase the entire block and then you can write data to it. 32 00:04:26.310 --> 00:04:29.340 William Cheng: Okay, so it's because it's kind of strange. That's all you have to do is that I said that here. 33 00:04:30.240 --> 00:04:40.710 William Cheng: Here's 512 kilobytes over here. And that's a block. Okay, in order for you to modify data inside this block. As it turns out, for the flash device you can only turn one into zero, but you cannot turn zero into one. 34 00:04:41.910 --> 00:04:51.450 William Cheng: Okay, so why did you have a zero there. You know, what was it a zero there there's nowhere for you to go go go back to one unless basically what we can. What we can do is that we can actually take this 35 00:04:52.620 --> 00:04:59.520 William Cheng: Take this block basically put it into an oven, and then we can bake it after we bake it for a while. Put assume everything will turn into a one. 36 00:05:00.990 --> 00:05:07.920 William Cheng: Okay, so I think on a flash device, you know, typically there are some kind of devices, their voters, maybe it's five volts or something like that in order for you to to cook the device. 37 00:05:08.130 --> 00:05:21.060 William Cheng: You have to, you actually have to apply something like 20 volt a 24 volts or 12 volts. So this way you raise the temperature of that particular blog and then everything will turn into why once they took out turn into one, then you can actually change anything upon want to a zero. 38 00:05:22.080 --> 00:05:24.390 William Cheng: Okay, so therefore you know what 39 00:05:24.900 --> 00:05:34.500 William Cheng: Of course, you know, that will you put things inside the oven. You're also damaging it. So, therefore, you know, when he tried to perform this program and he raised like oh you you raise a you programming erase a programmer. 40 00:05:34.710 --> 00:05:45.960 William Cheng: You can do this about 10,000 times then this blog. It's going to be useless because there will be ones that refused to go back. So they will be zeros that will refuse to go back to once once you have that was an entire block become useless. 41 00:05:47.130 --> 00:05:51.000 William Cheng: Okay, so this is why you know you know so. So imagine 42 00:05:51.450 --> 00:06:03.000 William Cheng: If you use it, use it as a drive and broad zero over here is your super blog. How often do you modify the superblock well you bought by some of our too often. So if you keep doing this. Pretty soon your super blog will be useless. 43 00:06:03.990 --> 00:06:09.180 William Cheng: Okay, so that's why we need to be very, very careful with devices like that. Yeah. All right. And also, if you 44 00:06:10.260 --> 00:06:18.810 William Cheng: If you don't program it if you just erase it and you know he raised it over and over again, even if you just do that. You can do it about 100,000 times and they also become useless. 45 00:06:19.650 --> 00:06:25.890 William Cheng: Okay. So, therefore, this is a sort of a very, you know, delicate, you know, a device. On the other hand, the hardware. 46 00:06:26.610 --> 00:06:33.690 William Cheng: On the hard drive. As long as you don't drop it. It's actually very, very, very, very reliable. Now what is, what is the solution to deal with this. 47 00:06:34.410 --> 00:06:39.630 William Cheng: So the solution over here is I want to make sure that the entire flight devices that they did the entire flash device. 48 00:06:39.990 --> 00:06:47.310 William Cheng: It's were out evenly across the entire flash device right over here. If you think about this in the flesh. Over here, right. So, so the idea here is that 49 00:06:47.790 --> 00:06:58.050 William Cheng: So again, the way I saw this is that there are these blocks over here blocks 01234, something like that. I'm going to make sure that for every blog. They are erased, about the same number of times. 50 00:06:59.070 --> 00:07:07.950 William Cheng: Okay, so this way. I'm not going to end up you know erasing only the first blog, you know, 10,000 times because the because in that case, they will become useless. Okay, so this is know as we're leveling 51 00:07:08.190 --> 00:07:21.900 William Cheng: I'm going to level the we're on the entire device so that every blog is, you know, is erased for for by the same number of times. Okay, so how do you actually do this right. So one thing that you can do this is that you know the basic idea here is a 52 00:07:22.980 --> 00:07:31.050 William Cheng: Is that the bay area over here that we're going to borrow the idea from the log structured process them because if you organize the entire disk as a log, then 53 00:07:31.470 --> 00:07:37.080 William Cheng: That is why I haven't. Right. It's a pen only and then it never modify. So, therefore, if the, you know, if this 54 00:07:37.320 --> 00:07:47.220 William Cheng: Device. Now look like this, right, if we use the flash over here using as log structured process them. Then in this case, we're doing a patent only. So, therefore, every plot is going to get written exactly only one time. 55 00:07:48.330 --> 00:07:59.640 William Cheng: Right when we reached the end over here. What do we, what do we reach the end right for the log structured process. Then we recently become a read only file says that, but for the flash drive over here, what we can do is that, you know, when it reached the end over here. So we have 56 00:08:00.990 --> 00:08:07.350 William Cheng: The luxury of bosses that you need to use the checkpoint file to keep track of where all your notes are so in the flesh. 57 00:08:07.950 --> 00:08:12.660 William Cheng: The flesh technology, the checkpoint fall where they call here. It's called a flash translation layer. 58 00:08:13.020 --> 00:08:23.940 William Cheng: Will get the translation layer is the one that keeps coming up for all the nodes are so what you can do that. You can go to the translation layer, figure out where all the notes are read them into memory and then you can erase the entire device. 59 00:08:24.780 --> 00:08:31.110 William Cheng: Day and day you you write all your fault. So the beginning the beginning part of your flash drive, and now you start using as a lot. Yeah. 60 00:08:32.490 --> 00:08:39.750 William Cheng: Okay. So, so what what if it doesn't really fit into memory. Okay, if there are too many files over here. So again, you could do this incrementally, you can take a bunch of files out, you know, 61 00:08:40.140 --> 00:08:49.620 William Cheng: Out of the beginning part of the flash drive, and then you can basically, you know, start writing to the flash drive, and eventually going to end up pushing everything to be in a flash drive. And now, again, it becomes a penalty until you 62 00:08:50.310 --> 00:08:57.570 William Cheng: To fill out the entire flash device. And then this way we can do this over and over again, you know, eventually is going to go to, you know, doing this 10,000 times 63 00:08:57.780 --> 00:09:03.150 William Cheng: Well, at that time, you'll flash drive of the flesh out become become useless but in this little but if you take this approach. 64 00:09:03.510 --> 00:09:09.930 William Cheng: Using the last retro forces them to take you a very, very long time before you write the entire device, you know, 10,000 times. Yeah. 65 00:09:10.590 --> 00:09:19.320 William Cheng: Yeah, so, so, so, so, yeah, the flesh, the flesh do over here. It's a read only. It's been only and never modify. So, therefore, you know the structure over here. 66 00:09:20.010 --> 00:09:28.080 William Cheng: Is a log structured process there. But what about the flesh translation layer, where do you put that you can't put that inside the flash device. The device, right, because if you do that in the flesh device. 67 00:09:28.290 --> 00:09:34.590 William Cheng: In the flesh. Translation. Translation layer is going to get modify all the time. Well, then in that case after 10,000 modification. It's also going to be dead. 68 00:09:35.220 --> 00:09:46.140 William Cheng: Okay, so the solution over here is that the flash translator, either they can go to a separate device. So if you have a hard drive and you have a flash drive some of our laptop right they have a hard drive. They can also you can also plug in a 69 00:09:47.760 --> 00:09:53.280 William Cheng: What do you call those things. The SDM devices or something like that, right. Those are the one I can see your camera. 70 00:09:53.610 --> 00:10:01.770 William Cheng: You can also put those kind of devices that the into. So in this case, you have a hard drive. So he said harddrive because store your flesh translation layer. So this way you can remember while you're rhinos are 71 00:10:02.070 --> 00:10:09.420 William Cheng: The other way to do it is that you can actually build that thing as such as the ranch houses them. You can feel the flesh, the flesh translation layer inside 72 00:10:09.780 --> 00:10:12.600 William Cheng: You know, he's a memory. So what you do is that, will you put your 73 00:10:12.780 --> 00:10:24.510 William Cheng: Will you put up your system. You got your you can scan the entire five try and find out where all the nodes are and what's the problem was I. Those are you can build a flash and sitting there and it's translation layer or the job of the transitional year is 74 00:10:25.110 --> 00:10:28.230 William Cheng: Is the map logical block number two physical block number 75 00:10:28.620 --> 00:10:34.320 William Cheng: Okay. So for example, when I tried to write to block number 20 right so again I can, I can keep track of all this in memory. 76 00:10:34.470 --> 00:10:46.560 William Cheng: So what I would do is I will allocate a new blog on the flash drive. So again, it's a penalty. So therefore, in this example, I need to find the last blog over here. So fits 65 is available. So instead of frustration there. I need to map. 77 00:10:46.920 --> 00:10:55.080 William Cheng: Create the mapping from one block number to another block number, right. So yeah, instead of flash especially their Wi Fi taco. I know numbers are this case we have you talk a lot block number 78 00:10:55.320 --> 00:11:08.940 William Cheng: So he's got a pressure and say here and there were no we need to remember block number 20 actually translate it into block number 65 on the physical device that and then later on when somebody tried to ask what block number 20 even though we are adding more stuff to 79 00:11:10.380 --> 00:11:18.780 William Cheng: The end of the log anytime somebody wants logical broke, I'm plenty. We go to the flesh translation layer and then we're gonna go to the right place inside of flash drive, and then get the data. 80 00:11:19.230 --> 00:11:22.110 William Cheng: There and eventually we're going to write to block number 20 again. 81 00:11:22.380 --> 00:11:32.040 William Cheng: Gay some point when the right time you get. And then in this case again with a pen only. So we're going to allocate a new blog over here. The New Black is going to be equal to 96 so now we're going to remember is that the flesh translation layer. 82 00:11:32.610 --> 00:11:42.660 William Cheng: Party needs to map the 96 right and this one, this one, all we try to read 20 you will go to 26 or you can also see that the old data over here, become useless. Again, that's the property of the logs right devices, then 83 00:11:42.990 --> 00:11:54.780 William Cheng: You end up with a lot of GARBAGE GARBAGE inside garbage inside a lot. Okay, so, so that's why we reached the end of the log over here. Yeah, plenty of things that you can clean up. And this way you know that the pastas, and what becomes smaller again. Yeah. 84 00:11:55.890 --> 00:12:03.180 William Cheng: So that's the basic idea. So the textbook. I just spent quite a bit of time talking about different kinds of a flash process then so 85 00:12:03.510 --> 00:12:12.240 William Cheng: I just sort of introduced this one of the higher level idea of how to do that. So if you're interested in detail. So all these things have an exit the bottom over here. You'll feel free to 86 00:12:13.320 --> 00:12:14.310 William Cheng: Feel free to read a textbook. 87 00:12:15.660 --> 00:12:21.480 William Cheng: For the last part of check this up to six. Are they briefly look at some of the, you know, you know, 88 00:12:22.890 --> 00:12:26.340 William Cheng: Some of the Linux and the the windows you know the process them. 89 00:12:27.090 --> 00:12:37.140 William Cheng: Linux had, you know, good. So at the time the textbook was written Linux has 57 file system built for Linux already well as even though. Today, we only when we use living will only see x two x. 90 00:12:37.590 --> 00:12:43.410 William Cheng: Three x, actually, for there has been a lot of, you know, fosters that has been built on Linux. Yeah. So, so 91 00:12:44.190 --> 00:12:54.870 William Cheng: If the file system is a very important verses that are because you know all the modern Linux system are built on top of the are based on the fastball system. Right. So remember I mentioned before he 92 00:12:55.560 --> 00:13:07.380 William Cheng: Is FFS calm, it's exactly the fastpass them UFC three is simply equal to two plus journaling, right. So, so this way. Yo yo yo yo your fastball system, you know, will be crushed zillion now. 93 00:13:07.920 --> 00:13:15.120 William Cheng: There's somebody named riser. I guess he really like be trees. So what he what he does is he goes to the the the 94 00:13:15.600 --> 00:13:19.980 William Cheng: The fastpass is then replace all the data structures that have houses that would be trees, but that's okay. 95 00:13:20.430 --> 00:13:29.100 William Cheng: You know what, we use a peach tree with a very, very large em, typically you required to levels that you can actually get to the data blocks. Right. So in this case, your overheads is only two the 96 00:13:29.550 --> 00:13:33.300 William Cheng: Two reads off on the desk and you can actually get to the data block. So we saw before. 97 00:13:33.660 --> 00:13:40.860 William Cheng: You know, for the for this is the VA system, the worst case is that you need to actually go to three reads and eventually get to the you go to data loss. 98 00:13:41.460 --> 00:13:51.690 William Cheng: For the Windows NT Fs. Again, it's going to be two weeks and then it will you do go to Data box. So by using the tree. The performance is going to be comparable to the empty file system. Yeah. 99 00:13:52.770 --> 00:14:01.980 William Cheng: So unfortunately, you know, riser. I heard that he got arrested. He went to jail. So he never finished this file systems and maybe one day when you get out of jail, we'll find out whether this file systems any good. 100 00:14:02.880 --> 00:14:10.740 William Cheng: And then yesterday for FC for is an improvement at St three on top of everything that yes, the three has to offer. They also they switch the data structure. 101 00:14:11.130 --> 00:14:23.370 William Cheng: To extend. So this way you can read to you know to to run this inside two different block and you can actually figure out what the date is. So again, he's going to reduce the the the reading from this about one disagree. Yeah. 102 00:14:23.790 --> 00:14:31.680 William Cheng: They also add a Logical Volume Manager is a UX team for they also use a different data structure to improve the performance of directories. So, there 103 00:14:32.010 --> 00:14:39.930 William Cheng: Are lots of data structure to choose from for UFC, for they choose a data structure called a hash trees. Again, we're not going to get into detail what history is now. 104 00:14:40.950 --> 00:14:47.430 William Cheng: Also Oracle as Oracle deck is there, the one that bought are some micro system. So my criticism that our businesses and that will assist them. 105 00:14:47.850 --> 00:14:57.270 William Cheng: They have a system called the BTR file system. Some people call it the better file system. Some people call it the butter file system. I have no idea which way it is now. But again, it's also based on the system. 106 00:14:57.630 --> 00:15:10.830 William Cheng: Okay, so. So again, this is why we covered this is the above us to Sam and then, you know, and then improving too fast. Our system and then they get sort of, you know, evolve into all the modern, you know, modern file system. 107 00:15:12.930 --> 00:15:18.330 William Cheng: When those empty. Again, we mentioned before they use the extent they also use the, you know, 108 00:15:19.860 --> 00:15:26.730 William Cheng: Different kind of be tree to optimize the directory. Look up. They also use journaling to provide crash resiliency. Okay. 109 00:15:27.090 --> 00:15:34.740 William Cheng: For Mac OS. As I mentioned before, initially they try to use, they were planned at all times are planning to use the CFS, which is based on 110 00:15:35.580 --> 00:15:46.110 William Cheng: The data that the shadow pages, but in the end they decide to use HDFS plus which use the extended data structure. They also use another feature called the p star tree. 111 00:15:46.560 --> 00:15:51.690 William Cheng: To improve the directory directory look up and then they also use generally to provide crashes resiliency. 112 00:15:51.960 --> 00:15:59.310 William Cheng: So we can see that out of all the CDS three to three system. They all have one thing in common is that they all use journaling to provide crash resiliency, right. 113 00:15:59.430 --> 00:16:09.630 William Cheng: And the reason for that is that journaling can be added to any file system that's not pressures of the end. And if you are generally to add, they all become crusher cilia. Okay, so it's a very, very popular way to, to, to provide question. So, yes. 114 00:16:14.880 --> 00:16:19.650 William Cheng: Alright, the last slide over here sort of show you the different layers you know inside the file system. 115 00:16:20.190 --> 00:16:25.260 William Cheng: That so so remember inside the process them right so let's take a look at the the middle part over here. 116 00:16:25.440 --> 00:16:33.660 William Cheng: The file system has two parts. One is the virtual file system and the other is the actual file system. So even inside of our system. Over here we have BFS. And we have the FS guy. 117 00:16:34.290 --> 00:16:45.540 William Cheng: So, you know, so, so, so, so yeah, so the VM FS is the one that provide the the abstraction that their files and directories and all these kinds of stuff around the lower level of here is going to try to implement all of these 118 00:16:46.920 --> 00:16:52.830 William Cheng: To implement all the, all these abstraction. So you can sort of consider the AFL actually include everything at the bottom over here. 119 00:16:53.280 --> 00:16:59.940 William Cheng: Now, all right. So what are the little bit. It's kind of weird. It's the buffer cache layer right the buffer cache layer over here, it doesn't really translate 120 00:17:00.390 --> 00:17:07.740 William Cheng: You know, from, you know, from, you know, like football club. I know number to some kind of a device address or something like that. The buffer cache is kind of a 121 00:17:08.970 --> 00:17:19.980 William Cheng: It's a layer that's purely there to provide better performance. Okay, so in this case the buffer cache. What it provides. Oh, here's a provide the abstraction that the device is actually very fast. 122 00:17:20.340 --> 00:17:28.740 William Cheng: Okay, we mentioned before, if the buffer cache has 99% he read it, on the average, you know, when you try to go to this is going to look like you're 100 you're, you're going to be 100 times faster. 123 00:17:29.100 --> 00:17:35.970 William Cheng: Than so this kind of layer over here, it can be slipped into the fastest and and provide additional functionality, such as improve performance. 124 00:17:36.300 --> 00:17:44.580 William Cheng: You also see other houses and what it will do is that I will add a layer for encryption. So this way you have more security you know, and things like that so so so 125 00:17:45.000 --> 00:17:52.950 William Cheng: That's what, that's one of the reasons the file system can be really complicated because people can actually keep adding layers to add functionality into into the system. 126 00:17:53.640 --> 00:17:59.310 William Cheng: Now alright the next layer over here is Logical Volume Manager. Again, we mentioned before, the Logical Volume Manager, you know, 127 00:17:59.670 --> 00:18:04.200 William Cheng: The top interface. Make it look like no matter how many this, you have it looks like just one giant desk. 128 00:18:04.380 --> 00:18:16.740 William Cheng: Okay, so therefore the Logical Volume Manager present multiple drive as one large array of blocks. Right. So again, the view of the of this is simply a rail blocks. So if you use a block number you can address any parties are your storage system. 129 00:18:17.310 --> 00:18:28.890 William Cheng: Okay, so if you're using a logical volunteer manager, you know, face. It's as if all you need is a block down, but you can get to any the number of days that you have and you don't really have to know how many days you have. Okay. It's a job with the logical manager to to to sort of 130 00:18:30.030 --> 00:18:42.720 William Cheng: Manage all these different discs that are sitting that that the all these different designs that are your physical devices now below the Logical Volume Manager over here. There's a block device interface. 131 00:18:43.770 --> 00:18:45.960 William Cheng: Now, you actually, you see that you'll convert to assignment. Right. 132 00:18:46.170 --> 00:18:54.120 William Cheng: There's a character devices. The block device, you know, Colonel. Something we typically deal with the character device, the character device interfaces read and write and things like that. 133 00:18:54.360 --> 00:19:02.100 William Cheng: What about a block device right we never have to deal with the blog device, but you can look at some of the kernel code, you will see that the block device in a weenie sister, the 134 00:19:02.730 --> 00:19:06.330 William Cheng: Device has two functions. One is to read a blog and the otherwise write a blog. 135 00:19:06.810 --> 00:19:11.880 William Cheng: OK. So again, the unit over here is a block and you can actually read a blog and write about from a particular device. 136 00:19:12.000 --> 00:19:19.950 William Cheng: So that means that over here the block device interface for us to talk to specific devices, right, so remember in the at the Logical Volume Manager over here, you only look at 137 00:19:20.880 --> 00:19:30.480 William Cheng: Your storage system is like one giant desk. Okay, one level below it, you actually you know about individual test and every, every individual this as a block device interface. Yeah. 138 00:19:30.930 --> 00:19:32.460 William Cheng: So the, so the block device interface. 139 00:19:32.790 --> 00:19:38.460 William Cheng: Or you know they did present the abstraction that the device is simply array of blocks are so now you have multiple devices. 140 00:19:38.640 --> 00:19:48.660 William Cheng: But every device is simply an array of boss. Yeah. And finally you get to the device driver, the device driver maps the block number two device addresses right again for a hard drive the divine addresses are what 141 00:19:49.020 --> 00:19:57.960 William Cheng: Are the dividers is our, you know, our head number the cylinder number in the sector number if the device is actually the flash drive. Well, in this case, you're going to get a block number, you know, for the flash drive. 142 00:19:58.440 --> 00:20:09.840 William Cheng: Guys, again, this is done in the device driver level. All right. And then, why go if you go below that, then, then you'll go to the hardware level you have the device controller and then you have the actual device out there are connected together. Okay. 143 00:20:11.370 --> 00:20:15.990 William Cheng: Alright, so again, instead of our system, you know, things can get a lot more complicated. OK. 144 00:20:17.070 --> 00:20:24.990 William Cheng: OK, so we are done with chapter six. So now we actually we're going to go back and take a look at Chapter four, chapter four, when 145 00:20:25.440 --> 00:20:27.840 William Cheng: I guess what was in chapter four. It's a simple opportunities, then 146 00:20:28.260 --> 00:20:39.060 William Cheng: Followed by a framework for devices. And then we skip the low level, Colonel. And the reason we skip that is because the Brown University code they wrote the device driver code. So therefore, we don't have to talk about the device driver until now. 147 00:20:39.780 --> 00:20:45.240 William Cheng: Yeah, we went over the process either thread right that's the one that go over the life cycles of processes in thread. 148 00:20:45.660 --> 00:20:55.950 William Cheng: We also introduce you know give you a high level, you know, Introduction to the storage management and now we're going to come back and look at, you know, look at the design decision you have to make when you try to design a device driver. 149 00:20:56.640 --> 00:21:05.760 William Cheng: Okay, so we're going to talk about exactly how they implement a device driver, because a lot of stuff. It's very hardware your specific IP, you know, so, so we're going to talk about the higher level design issues. Okay. 150 00:21:07.770 --> 00:21:09.030 William Cheng: All right, we're going to talk about 151 00:21:09.300 --> 00:21:17.490 William Cheng: Two different devices. One is the terminal device that determine that you have been using all this time right when you are whenever you run Ubuntu 16.04, you have to terminal. 152 00:21:17.640 --> 00:21:28.980 William Cheng: So, so it looks like you're talking to a terminal device. So now we're going to see what you know what the device driver for that terminal device look like. Okay, and we're going to sort of briefly take a look at the network communication. So, again, how do you communicate over the network. 153 00:21:30.120 --> 00:21:37.500 William Cheng: Okay, well you said to communicate over the internet. What do you do while you open you read and write and you coast. If I exactly the same way as you talk, I'll talk to you know 154 00:21:38.040 --> 00:21:42.750 William Cheng: Talk to the terminal device. So in a way, the terminal device and network communication device. They're very, very similar. 155 00:21:43.710 --> 00:21:54.630 William Cheng: Okay, the interface level. They're all similar. So we're all going to see what makes them different. Right. I'm going to briefly talk about how do you adjust the issue that makes the network communication devices, very different, you know, from the terminal device. Yeah. 156 00:21:56.250 --> 00:22:07.320 William Cheng: Alright, so here's a terminal. If you blow this one up over here, you'll see down the left hand side says digital. So this is the terminology that that's embedded by the Digital Equipment Corporation. On the right hand side it says 157 00:22:08.730 --> 00:22:10.920 William Cheng: This is one of the most popular terminal. 158 00:22:11.310 --> 00:22:15.810 William Cheng: You know, for a mini computer. So these are common core, they are the one that embed a mini computer 159 00:22:15.990 --> 00:22:24.720 William Cheng: The mini computers are the facts 11 seven at that we saw that the size of a refrigerator and things like that. So, this is this is going to be the console in front of the facts 11, seven, eight. 160 00:22:25.050 --> 00:22:32.490 William Cheng: Okay, so, so, so the abstraction, you know, for using such a terminal is that, you know, when you're sitting on a terminal. I guess you're using you're using character graphics 161 00:22:33.030 --> 00:22:38.160 William Cheng: On the screen over here is going to be 80 characters across and then about 24 rolls on the screen over here. 162 00:22:39.120 --> 00:22:43.020 William Cheng: Now, and also, you know, this particular device, you know what 163 00:22:43.710 --> 00:22:49.080 William Cheng: Is physically connect your machine, right. So over here on the left hand side is going to be the vaccine 11 seven at machine over here. 164 00:22:49.320 --> 00:22:57.870 William Cheng: And the terminology here is going to be physically using a serial cable connect to the fax machine. So the back of the fax machine. There's gonna be a serial port. 165 00:22:58.380 --> 00:23:09.060 William Cheng: If you have ever used a Microsoft. If you have ever used to use a PC, especially in the good old days, there is a connector at the back over here called a serial connector so zero connector is 166 00:23:09.690 --> 00:23:15.960 William Cheng: A sort of connects to the serial port of your device. So in this case, you're going to use the serial cable a serial cable, you know, sort of look like this. 167 00:23:16.230 --> 00:23:24.960 William Cheng: It's kind of like a like a trap is I shaved. There's a bunch of holes over here or two rows. So that's the, that's a serial port. 168 00:23:25.380 --> 00:23:29.760 William Cheng: Okay, so when you have a terminal. You are physically plugged into a syrup or the bag over here. 169 00:23:29.910 --> 00:23:38.400 William Cheng: So every key press that you press over here is going to generate and interrupt inside you know inside this machine, right, because again this is physically connected. So every time we press the key. 170 00:23:38.610 --> 00:23:43.170 William Cheng: There is a device controller for sittings on the machine. What it will do is that every, every key that you press 171 00:23:43.440 --> 00:23:51.870 William Cheng: What it does regenerating interest and he said he Rob servers with D. What it will do is it will come to that this particular device, say, hey, which key you just press and then it will copy this. The 172 00:23:53.040 --> 00:23:54.900 William Cheng: This particular character into the memory. 173 00:23:55.800 --> 00:24:04.770 William Cheng: That. And also, this is a very slow device when you try to put character onto the onto the screen, you need to actually put the character on the screen one character at a time. So would you do that you'll put 174 00:24:05.310 --> 00:24:08.160 William Cheng: And then clear this up a little bit, right, you were going to put one character on 175 00:24:08.400 --> 00:24:18.150 William Cheng: You know, on the device over here and now the device is busy. You have to wait for this character to finish, I put into character on the device. So in this case you will your kernels. I will go to sleep. Wait for interrupt when it 176 00:24:18.510 --> 00:24:23.460 William Cheng: Happened in Iraq service which routine, you're going to start the next I will operate you you put the next character onto the screen. 177 00:24:24.210 --> 00:24:30.630 William Cheng: Okay, so the character. I'll put on the screen one character per interrupt. Okay. Until, and then you can actually put a character. Character over here. 178 00:24:30.810 --> 00:24:37.080 William Cheng: Is backslash and it's gonna go through next role even Bashar. It's all over here, it's gonna go to the beginning of the lie, and then you put the next slide here cetera 179 00:24:37.620 --> 00:24:42.210 William Cheng: Okay, so that's the kind of device that you're dealing with. When you're using a terminal device. Okay. 180 00:24:42.840 --> 00:24:49.260 William Cheng: So this guy is what kind of device drivers or should we use that as it turns out, since this one is connected to the serial port device. 181 00:24:49.710 --> 00:24:59.730 William Cheng: For every machine. There's actually a single device. There's a serial port driver. Okay. The serial port driver. As it turns out, is totally unsuitable for the terminal device. 182 00:25:00.720 --> 00:25:11.670 William Cheng: So, so in that case, you know, we actually, we need to write a special driver, you know, for the terminal. Now the reason for that is that the serial port po po po po po. The the 183 00:25:12.480 --> 00:25:18.930 William Cheng: What is it called the sewer line driver over here the sewer line drive where we are typically what it can do that, you know, we can only handle one character at a time. 184 00:25:19.170 --> 00:25:25.200 William Cheng: Okay, so when you're typing. If you type one character you're going to generate a interrupt if it turns out that the the operating system. 185 00:25:25.500 --> 00:25:33.240 William Cheng: Is a servicing high level interrupt will have the interrupt block or disable or something like that, while you're not case, you can actually press. The second key will wipe out the previous key. 186 00:25:33.720 --> 00:25:38.820 William Cheng: Okay, so if you're using a Syrah line driver and if you type really fast, you're going to end up missing a lot of keystrokes. 187 00:25:39.480 --> 00:25:45.450 William Cheng: Okay, and also when you turn it this way character onto the screen if you try to send too many characters to this to the to the zero line device. 188 00:25:45.660 --> 00:25:53.280 William Cheng: To a certain line device. Why not case, you know, some of the character over here will be wiped out. So therefore, on the screen, you only see some of the character you try to print onto the screen. 189 00:25:54.240 --> 00:25:58.500 William Cheng: Okay, so therefore the serial entrepreneur basically is the only good for, you know, dealing with the 190 00:25:59.040 --> 00:26:05.700 William Cheng: You know the dealing with a device that's really, really slow or, you know, the human needs to be aware of and type on the keyboard. Very, very slowly. 191 00:26:06.240 --> 00:26:15.660 William Cheng: Okay, so, so if you're taking. So, so you're taking a class that you're building a computer. Sometimes you end up doing the deal. The serial device when you try to deal with the sewer divide, you got to be very, very slow. 192 00:26:16.020 --> 00:26:18.030 William Cheng: Because otherwise, some of the character will get wiped out. 193 00:26:18.540 --> 00:26:26.100 William Cheng: Okay so clearly for a modern computer, you know, you should be able to type as fast as you want it should also be able to, you know, write it out to the screen as fast as you want. 194 00:26:26.490 --> 00:26:33.810 William Cheng: Okay, you don't really want to wait for this, you know, one character so so basically you don't want every operation to be a synchronous operation right because in that case it will be too slow. 195 00:26:34.410 --> 00:26:46.170 William Cheng: Because this guy is we need to build a special, special device driver to replace the serial line driver in case we are connected, you know, in case we're connected to a to a terminal that 196 00:26:47.490 --> 00:26:57.840 William Cheng: Oh, I better over here. There's also the current or comment or your city. Oh, well, you know, we're sitting in front of terminology abstractions that you're physically connected to your machine, right. So, therefore, this one is the device console. 197 00:26:58.080 --> 00:27:05.730 William Cheng: For your machine or if you have the right privilege. If you know the password for the super user. Once you log in as a super user, you can actually shut down the machine. 198 00:27:06.750 --> 00:27:12.690 William Cheng: Okay, so again this is this kind of connection is very, very different from a web connection where you have a webcam and actually you're connected to Google. 199 00:27:12.900 --> 00:27:23.610 William Cheng: Can you shut down, Google, of course, you cannot. Okay. So yeah, there's kind of a terminal connection, the implication over here that you're physically connected machine. OK. So again, it's a very different kind of different kinds of different kind of connection. 200 00:27:25.110 --> 00:27:35.190 William Cheng: All right, so, so, so, you know, when you start to implement a tournament or device driver, you need to be need to be a little bit spy device specific right because we know that if we type too fast or whichever 201 00:27:35.640 --> 00:27:38.280 William Cheng: Way too fast, the CEO line driver is not going to cut it. 202 00:27:38.550 --> 00:27:43.290 William Cheng: Okay. So in this case, I mean, but but we sort of we talked about it when we tried out right obviously some co 203 00:27:43.440 --> 00:27:51.780 William Cheng: We want our opportunities and to be as device independent as possible. Right. So right now we're advocating to say that while maybe for the terminal, we actually have to be device specific 204 00:27:52.530 --> 00:27:57.030 William Cheng: Okay, so in this case the quite the Fair question to ask is how device specific, do you want to be. 205 00:27:57.660 --> 00:28:03.120 William Cheng: Okay, so we still want to implement as the terminal device drivers or in a more at device independent way as possible. 206 00:28:03.450 --> 00:28:11.340 William Cheng: So we're going to sort of try to divide the time to sort of figure out some of the issues are going to be device independent and some of the issues are actually going to be device dependent now. 207 00:28:12.420 --> 00:28:14.190 William Cheng: So the problem with terminal. So again, that's one of the 208 00:28:14.670 --> 00:28:27.990 William Cheng: That's one of the design decision over here is a how to actually divide implementation of the terminal device driver so that part of it can be independent of the manufacturer of the device and some of them can be can can can actually be device independent. Yeah. 209 00:28:28.830 --> 00:28:31.830 William Cheng: The issue for terminology here is the one that I mentioned I mentioned before. 210 00:28:32.280 --> 00:28:39.780 William Cheng: Okay, so, so, so the terminals are too slow, and the character generations are too fast, right. So when he tried to print stuff onto the screen over here. 211 00:28:40.560 --> 00:28:51.210 William Cheng: You know, we want to be able to slow things down. Okay. So in this case, what we can do is that we can actually use an output buffer. So when you try to print something out to the display. There's going to be a buffer that will store a bunch of data. 212 00:28:51.690 --> 00:28:56.970 William Cheng: Okay, but we already seen the buffer like that right that's known as the producer consumer problem. So you can actually sort of see 213 00:28:57.150 --> 00:29:03.510 William Cheng: See, I'm going to sort of turn the picture you know 90 degrees over here. So in this case we have a producer at the top over here in the consumer at the bottom. 214 00:29:03.720 --> 00:29:10.170 William Cheng: The consumer is the ones going to put the character onto the device. Okay, who's going to get a producer what the producer is going to be your 215 00:29:11.340 --> 00:29:19.590 William Cheng: Producer is going to be your application. Well, your application. Make the rice system call you're going to start putting data into this producer consumer problem, right. So the good thing. 216 00:29:20.280 --> 00:29:32.340 William Cheng: So why do we use a producer producer consumer problem, right, if the problem device over here is too slow. We tried to pull the data out to the screen. Very, very slowly. Pretty soon this buffer will get full when above will get for the producer will get blocked. 217 00:29:33.390 --> 00:29:40.800 William Cheng: Okay, so therefore using a producer consumer problem to studio to to to to to to to deal with a slow device. It's actually really, really good way to go. 218 00:29:41.400 --> 00:29:46.110 William Cheng: Okay, so therefore, instead of Colonel, we're going to use a buffer that's implemented using a producer consumer problem. 219 00:29:46.410 --> 00:29:54.150 William Cheng: So this way we can actually write data out to the buffer as fast as we want, while the data is going to get pulled out from the buffer and sent to the display, you know, very, very slowly. 220 00:29:55.020 --> 00:30:02.190 William Cheng: But whenever the character from the keyboard over here, you know, they can also be generated pretty fast, right, you're typing on the keyboard. 221 00:30:02.640 --> 00:30:11.280 William Cheng: Can your computer, keep keep up with the speed of your typing. I mean, yeah, if you think about this is that of course the computer is really fast. You know, when I keep typing over here, you know, 222 00:30:12.450 --> 00:30:19.050 William Cheng: will will will not keep having over here, the computer, you know, the, the, the, the computer should be able to keep up. 223 00:30:19.410 --> 00:30:27.390 William Cheng: Okay, but please remember that, you know, you only get the character from the keyboard. When you make the system call right if you don't make the system call, you're not going to get the character. 224 00:30:28.140 --> 00:30:35.640 William Cheng: You're not gonna get a character on the keyboard. So therefore, the character is going to start accumulating inside the Colonel, you keep typing and typing the more character you type 225 00:30:35.880 --> 00:30:43.620 William Cheng: All these character is going to is going to be stored inside the colonel okay if your application number called we none of these characters going to get delivered to the application. 226 00:30:44.190 --> 00:30:52.770 William Cheng: Right, you haven't used to sort of done that in warm up to. I won't be running a simulation for a long time. If you start typing ahead you know all the characters, sort of, you know, sort of the 227 00:30:53.160 --> 00:31:01.470 William Cheng: They all went into Colonel, but they never got delivered into application, right, when did they get delivered your application when you finished running one or two. And now you go back to your 228 00:31:02.730 --> 00:31:06.540 William Cheng: You go back to your logging shell and your luggage our start taking your commands. 229 00:31:07.560 --> 00:31:17.640 William Cheng: Okay, so therefore again inside the Colonel, you need to be able to store this data. So therefore, we also need to input buffer is that occur and also the input over here also implement as a, you know, as a producer consumer problem. 230 00:31:17.820 --> 00:31:27.780 William Cheng: So in this case, the producer over here is going to be so who's going to be a producer. We as user typing characters all these characters going to get produced and they will go put into the circular buffer. 231 00:31:28.020 --> 00:31:34.740 William Cheng: And then who's going to be the consumer the consumer is when the application made the system call and then it will take data out of this buffer. 232 00:31:35.640 --> 00:31:39.150 William Cheng: There. So therefore, instead of Colonel, we're going to see the picture. So look like this. 233 00:31:39.420 --> 00:31:42.390 William Cheng: Okay, so in this example here, this picture, we actually have two different 234 00:31:42.600 --> 00:31:53.310 William Cheng: device driver. One is the keyboard device driver at the otherwise that display device driver, because typically these days, we are using USB devices. So therefore, the keyboard and the display. There are two separate devices. 235 00:31:53.850 --> 00:31:55.980 William Cheng: But in the globe as they are combined into one. 236 00:31:56.430 --> 00:32:00.960 William Cheng: One device driver. Okay. So whether you know they are one device driver or to divulge it doesn't really matter. 237 00:32:01.140 --> 00:32:09.270 William Cheng: What's important is that inside the corner over here. We're going to have to buffer wonder if the alpha buffer. And the other one is improper right and both of them I implemented using a producer consumer problem. 238 00:32:10.470 --> 00:32:13.980 William Cheng: OK. So again, I mentioned before, the president consumer problem. They are very, very 239 00:32:15.180 --> 00:32:25.770 William Cheng: Very useful and they're, you know, they're implementing using some of for so again this is, you know, this can be think of as a pipeline Apollo is that you have, you know, one producer over here is sending data to the console. Yeah. 240 00:32:27.090 --> 00:32:34.650 William Cheng: Alright, so let's take a look at that. You know that this pretty good picture over here, right. If we look at the north side that the top side all of these to buffer. 241 00:32:34.980 --> 00:32:38.490 William Cheng: Who's interacting, you know, with the with it with alpha buffer in the 242 00:32:39.060 --> 00:32:42.450 William Cheng: Upper right so again for the output buffer is that what your application program. 243 00:32:42.630 --> 00:32:51.420 William Cheng: Made a racist them call, then you're going to write data, your data into the upper right. So when you make the racism, call your user said become a colonel threat right it comes out of current over here. 244 00:32:51.600 --> 00:32:58.710 William Cheng: And then what it would do is that it will attempt to write data into the buffer. If the buffer is full, that in this case the colonel says gonna fall asleep. 245 00:32:59.970 --> 00:33:09.210 William Cheng: Okay, so, so good you know if the device over here is really, really slow your cursor is gonna fall. See, so on the top side over here again is the user to add that becomes the colonel threat and they're you they're interacting with the pop side. 246 00:33:09.510 --> 00:33:16.620 William Cheng: Of the of the of upper right. What about the info buffer over here, good input over here again the top side over here. It's going to be the user 247 00:33:17.130 --> 00:33:20.850 William Cheng: Will you make the system called the user has not going to the current over here, become a counselor. 248 00:33:21.030 --> 00:33:28.830 William Cheng: And the Colonel. Third is going to be the consumer that will take data from the input buffer and then it will do is that it will return it back into the back end user space with the data. 249 00:33:29.250 --> 00:33:36.540 William Cheng: Okay, so if your application is not making the making the racism call all the characters over here is I'm a guest stuck inside info buffer over here. 250 00:33:37.200 --> 00:33:45.330 William Cheng: So in that case, what will, what's going to happen. I don't know if you have noticed that if you keep typing a p typing at some point. The keyboard is going to start beeping 251 00:33:46.380 --> 00:33:57.180 William Cheng: Okay, so why is the keyboard stopped BB the keyboard stopped BP is telling you that you know this particular producer consumer problem that buffer is completely full. So all the new character that you type has been lost. 252 00:33:58.920 --> 00:34:06.630 William Cheng: Okay, so it is possible. You know, for the input buffer over here to lose character, but what about the output over here, the output buffer can never lose any characters. 253 00:34:07.350 --> 00:34:08.460 William Cheng: Okay, because Robin is though. 254 00:34:08.820 --> 00:34:17.520 William Cheng: If you're, if you're if you're sending data into the buffer and a to hire overlay. Why not case, you know, your kernel says going to get blah, so therefore you stop generating character into the upper buffer. 255 00:34:17.730 --> 00:34:21.420 William Cheng: So this way, every character that you print out onto the screen, they will all show up on the screen. 256 00:34:22.110 --> 00:34:28.080 William Cheng: Okay, which kind of makes sense, right, for the upper buffer. You know when you try to display character onto the screen. You don't want to miss a single character. 257 00:34:28.260 --> 00:34:33.270 William Cheng: But why you are you're providing input to your program when you're typing away. You know, is it okay to lose character. 258 00:34:33.480 --> 00:34:40.740 William Cheng: Well, it's okay to lose character as long as the keyboard beep at you right with the Kibo P i p value we are trying to say that, oh, I'm losing characters. 259 00:34:41.250 --> 00:34:50.040 William Cheng: Okay, so in that case you will be allowed to actually for the operating system introduce characters to do to lose the character you type because when you hear start up yourself typing right so this way, they will, you know, 260 00:34:50.700 --> 00:34:57.120 William Cheng: Eventually you application over here where we data on the input buffer. And then this way, the new character that you tie will be able to go into the input offer 261 00:34:58.110 --> 00:35:05.100 William Cheng: Because even though the info buffer and alpha, they're all, for instance, or producer consumer problem, but the characteristic over here is completely different. Because 262 00:35:05.340 --> 00:35:14.160 William Cheng: Because, you know, when we try to print character onto the screen. We want to make sure we never lose a single character. Wow. Typing on the keyboard. It's okay to lose character. If you start your beeps. Yeah. 263 00:35:14.850 --> 00:35:24.630 William Cheng: Alright so I'm synchronized with the fall 2019 lecture over here. So I'm going to end this lecture right now, so I will see you until election it. Okay.