WEBVTT

1
00:00:03.179 --> 00:00:22.920
William Cheng: Okay, this is the second part of lecture 11 so we just finish the, I guess the the interrupt stuff. So now we're going to go to chapter four, chapter four, at the beginning, we, I guess we would talk about the framework for devices and

2
00:00:24.090 --> 00:00:39.510
William Cheng: We also talk about processes and threads and we skipped the low level, Colonel. We're going to come back to it after we're done with chapter seven. So now we're going to sort of look at the still kind of introductory you know stuff for storage management. Okay.

3
00:00:40.860 --> 00:00:49.620
William Cheng: All right. So, so what kind of, you know, physical device can you use to store data. So there are two kinds of two places you can store data. Number one is the

4
00:00:50.400 --> 00:00:59.520
William Cheng: Is the physical memory and number two is the is a desk that you can store data. So again at the typical terminology is that

5
00:01:00.120 --> 00:01:10.860
William Cheng: You know, the, the, the physical memory is no as primary storage, because that's how you, how you program mainly store their data. You put in your address space your address basis, map the physical memory.

6
00:01:11.130 --> 00:01:14.160
William Cheng: So physical memory is known as primary storage that

7
00:01:14.610 --> 00:01:20.820
William Cheng: The disk space storage. So when we talk about this. We'll talk about, you know, all kinds of this and all kinds of storage devices.

8
00:01:21.060 --> 00:01:26.670
William Cheng: Could be a hard drive, could be a USB stick could be whatever other in a storage devices that you have there.

9
00:01:27.090 --> 00:01:37.770
William Cheng: So these are known as a secondary you know storage. So, so when we sort of think about it is that they're kind of like, you know, backup of your primary storage, guys. Oh, sorry. Again, it's called secondary storage.

10
00:01:38.250 --> 00:01:43.170
William Cheng: The primary storage. They are consider they're considered to be directly addressable

11
00:01:43.500 --> 00:01:51.270
William Cheng: Yeah, even though we are using one level interaction, right. So again, what is the one level direction where we use a page table to map virtual address and physical address

12
00:01:51.690 --> 00:01:54.600
William Cheng: But this is done so quickly, because this is done in hardware.

13
00:01:55.140 --> 00:02:00.480
William Cheng: So therefore, it's going to look like you know that you are you know that you are able to access the physical memory directly

14
00:02:00.810 --> 00:02:05.520
William Cheng: There, although we're going to one level in direction. But again, this is Danny hardware. So therefore superfast

15
00:02:06.210 --> 00:02:14.790
William Cheng: Yeah, and also another common over here. So, again, to remember that you don't have three different types types of hardware. We have the CPU where the memory and everything else. We call them devices.

16
00:02:15.060 --> 00:02:20.730
William Cheng: Right over here. So in a way, you know, physical memory is really not a device over here. So what physical device can you use to store data.

17
00:02:21.030 --> 00:02:26.730
William Cheng: So, so basically just talking about, you know, something that sort of looks like at the moment feel like a device, but please understand that memory.

18
00:02:26.940 --> 00:02:33.270
William Cheng: Is is quote unquote directly accessible so therefore it's not considered a device, right, so what's the device devices, what you

19
00:02:33.930 --> 00:02:42.330
William Cheng: To access a device, you need a device driver. I mean, clearly when you try to access memory. You don't need to go through a device drivers, so therefore memory is not a device.

20
00:02:43.710 --> 00:02:52.920
William Cheng: To go to the secondary stories to go to a display stories, you need a device driver. So there's so so so again, there, there are two things that you can do with the secondary storage. One is

21
00:02:53.610 --> 00:02:58.980
William Cheng: Is to store files that were mentioned before they're just abstraction know files and files that is an array of bytes.

22
00:02:59.400 --> 00:03:07.710
William Cheng: Was there a briefly talk about that in chapter one, right, so, so, so, so those kind of files are stored in a storage device. Now we can also use a second

23
00:03:08.100 --> 00:03:12.750
William Cheng: secondary storage device to support virtual memory again to create the illusion that

24
00:03:13.650 --> 00:03:20.010
William Cheng: That you know that we're four gigabytes or address space, even though, physically, we might not. We would might not have that much memory.

25
00:03:20.760 --> 00:03:28.590
William Cheng: Now, so, so, so again reminder that the application, you know, when you're thinking about your application, you know, if you want more money, along with a warm up to

26
00:03:29.070 --> 00:03:37.890
William Cheng: They are only aware virtual memory. Okay, so, so, so, so will you read your calling warm up on a warm up to, you know, do you ever think about physical memory, what

27
00:03:38.460 --> 00:03:43.050
William Cheng: So actually, you know, you probably you think about physical memory. But you think that the virtual memory is physical memory, right.

28
00:03:43.590 --> 00:03:48.810
William Cheng: Okay, so there's really, you know, so we write your application. There's no concept of physical memory everything

29
00:03:49.290 --> 00:03:53.490
William Cheng: Even though you know me, I think there is. But in reality, everything is virtual memory now.

30
00:03:54.060 --> 00:04:01.320
William Cheng: The application should not care about. I mean, that's the whole point of, you know, having optimism support is that, you know, people who write applications, they don't

31
00:04:01.560 --> 00:04:08.910
William Cheng: Really have to worry about how much physical memory is available to you to you otherwise. And if you think about what you try to run your array warmer and warmer to

32
00:04:09.420 --> 00:04:14.340
William Cheng: You know, you might have to write your, your program differently if you have small amount of memory or large amount of memory.

33
00:04:14.850 --> 00:04:22.350
William Cheng: So in that case is going to be very difficult for you to write program when long, long time ago. You know, people have, you know, when people write write software they have to worry about that. But, you know,

34
00:04:23.100 --> 00:04:29.580
William Cheng: Over the years of operating systems will take over. So you don't have to worry about that right, all you need to worry about is that how big of address space you have.

35
00:04:29.760 --> 00:04:36.180
William Cheng: Your four gigabyte address space. We have a 32 bit CPU and you have a much, much bigger address phase when you have a 64 bit CPU.

36
00:04:37.140 --> 00:04:47.130
William Cheng: Yeah, so there may not be enough physical memory for all the processes, right, if you add up all those sort of the memory usage for all the 50 different processes that are running on your laptop, your desktop.

37
00:04:47.640 --> 00:04:51.180
William Cheng: That might be bigger than the physical memory and there's no problem running all the software.

38
00:04:51.660 --> 00:04:59.100
William Cheng: Okay, because the operating system is using the virtual memory technique over here to hide all the details away so so application program never have to worry about it.

39
00:04:59.460 --> 00:05:10.650
William Cheng: Okay, so that becomes the worry, you know, for the operating system. Yeah, the ordinances and make sure that the real physical storage as the real primary storage is only available when necessary.

40
00:05:11.220 --> 00:05:19.020
William Cheng: Okay, well, your program needs. You know when you provide needs physical memory, then it's a job, the ordinances and the provider for you.

41
00:05:19.650 --> 00:05:27.210
William Cheng: Guys okay in chapter seven. We're going to talk about the detail how to do that. And as it turns out, you know, your file system, your storage system to actually get involved.

42
00:05:27.630 --> 00:05:31.890
William Cheng: So in chapter four sort of talk a little bit about, you know, how, how that actually works. Yeah.

43
00:05:32.580 --> 00:05:41.370
William Cheng: Right, here's an example application, you know. So let's say that your machine only has 256 megabytes of memory. I mean, these days, we have a lot more memory than that. I mean, you know,

44
00:05:42.360 --> 00:05:48.660
William Cheng: In the good old days, you don't have too much memory. Can you allocate a one gigabytes block of memory, you know,

45
00:05:50.280 --> 00:05:53.730
William Cheng: You know forth for a 32 bit for for 32 bit process.

46
00:05:54.150 --> 00:06:03.120
William Cheng: Right. The answer is yes. Right, because the abstractions that you have four gigabytes of virtual address space if you want to, you know, if you want to allocate one gigabytes of memory. You're allowed to do that.

47
00:06:03.990 --> 00:06:10.290
William Cheng: Okay, you don't have to worry about that you don't have enough physical memory. Okay. Because the organization is doing all this kind of stuff for you. Yeah.

48
00:06:11.310 --> 00:06:16.650
William Cheng: All right, so yeah I draw this picture right every program things that they have a 32 bit address may have four gigabytes.

49
00:06:16.920 --> 00:06:31.380
William Cheng: Parts of em, you know, using the virtual memory maps parts of a mind map the physical memory. The rest of the map to the desk right the map the secondary storage device. And, you know, if you try to access, you know, data that are mapped to primary storage then then then

50
00:06:31.890 --> 00:06:43.530
William Cheng: So in that case, you get to use it without any interruption. But as soon as you try to access data that are mapped to the desk or in that case, you know, you're going to try to open it says that the evidence isn't going to kind of take the data copy them into

51
00:06:44.130 --> 00:06:49.230
William Cheng: The physical memory and then change the virtual memory map over here. So, so you can continue to use it.

52
00:06:50.070 --> 00:06:57.630
William Cheng: OK. So again, that's the high level concept that we talked about in chapter one. Yeah. So, so there's a fancy data structure inside the opposite of them will call it the virtual memory map.

53
00:06:57.900 --> 00:07:06.900
William Cheng: And it's complicated. You know, because it's part hardware part operating system. So, so we already saw part of it. Right. There's the, you know, there's the

54
00:07:07.470 --> 00:07:13.020
William Cheng: The page table right the other parts of it. So today I want to see a little more of that that that that that picture. Yeah.

55
00:07:13.530 --> 00:07:20.760
William Cheng: So, so we when we implement the virtual memory system where we use the virtual memory map every program things it has four gigabytes of address space.

56
00:07:21.000 --> 00:07:33.210
William Cheng: Okay. And if you try to use them. You know, if you if you actually allowed to access them. The opposite will create a memory for you so you can so you can use as much memory as you want. Okay, even though physically. We don't have enough memory. Okay.

57
00:07:35.220 --> 00:07:46.530
William Cheng: Alright, so the truest for the application to access the secondary storage. So one is to basically use the file system. Right. So again, we mentioned that the two different ways. One is called sequential i o the other ones called block I O

58
00:07:46.950 --> 00:07:54.690
William Cheng: So the system called that you may put a sequential I O is going to open a file and it will return to you a file descriptor. And then you can call read and write. And then we're done with the call clothes.

59
00:07:54.960 --> 00:08:01.800
William Cheng: Made. The other way is to do a block I O and we mentioned before in last time we'll be talking about Chapter four, is that you can map a fine to address space.

60
00:08:02.070 --> 00:08:08.850
William Cheng: So in that case, you still use open and close. Right. So you open a fall over here. I don't know if you actually look at the man page for em map.

61
00:08:09.180 --> 00:08:14.010
William Cheng: Guys, I'm after the system call when he used a map system call one of the argument is the file descriptor.

62
00:08:14.370 --> 00:08:22.230
William Cheng: Okay. So, therefore, you know, that means that you have to open a file. First you got a file descriptor. And then you pass the file descriptor to at the end map system call and he can actually map.

63
00:08:22.440 --> 00:08:28.620
William Cheng: You know, you'll fall into. I just based you come out the entire file into outer space, or you can map a portion of the file into the address space.

64
00:08:29.280 --> 00:08:38.730
William Cheng: Then. Alright, so, so, so I guess lado and Colonel three. What is it we're talking about more details. It's in chapter seven, when talking about more detail and also in

65
00:08:39.300 --> 00:08:46.140
William Cheng: Your kernel three assignment you actually have to implement that. Okay, so, so, so by the end of semester, you're going to know this kind of stuff really really well.

66
00:08:46.950 --> 00:08:57.660
William Cheng: Right, so, so some of those sort of memory management concerns are, you know, how do you really map virtual address real one, we sort of saw that already right using you're using a page table and things like that.

67
00:08:58.080 --> 00:09:05.250
William Cheng: You also need to determine which addresses are valid. So they are referred to allocate a memory and which virtual just yet. No, you're not allowed to use

68
00:09:05.490 --> 00:09:13.440
William Cheng: Okay, we need to keep track of, you know, which real object, if any, are mapped into the range of the virtual address. And also, which one are not mapping mapping to

69
00:09:13.800 --> 00:09:19.050
William Cheng: The to the range. Also, in that case, when you try to access those memory location, then in that case we need to kill your program.

70
00:09:19.560 --> 00:09:28.410
William Cheng: Okay, it's a part of the operating system responsibilities that, for example, if you access memory location zero what is happening isn't supposed to do the opposite of shouldn't really create memory for you.

71
00:09:29.010 --> 00:09:35.160
William Cheng: Okay, so what the offices of job is to kill your program guys will get all these are done as part of the virtual memory system. Yeah.

72
00:09:35.520 --> 00:09:47.340
William Cheng: And also when you start running on a physical memory, then it's obviously some job to do, to, to, to, to decide what to keep in the primary storage, right. So again, this is, you know, it's called ram or physical memory. And also, you know,

73
00:09:48.510 --> 00:09:54.120
William Cheng: What the fetch from somewhere else. So if you need to, you know, take your memory as story into the secondary source.

74
00:09:54.600 --> 00:10:05.400
William Cheng: Put into secondary storage, where do you put them. Okay, do you put them inside the file system or you do put it somewhere else. OK. So again, you know, they'd only in chapter six. We're also going to talk about exactly how this is done.

75
00:10:07.140 --> 00:10:13.110
William Cheng: Alright, so as we mentioned before, you know, we're going to use the page table over here to map virtual address to physical address

76
00:10:13.680 --> 00:10:21.450
William Cheng: So so so the message over here. Is that a valid virtual address must be ultimately resolvable by the operating system to a location physical memory.

77
00:10:21.750 --> 00:10:23.430
William Cheng: Okay, so this is actually done in two steps.

78
00:10:23.670 --> 00:10:30.960
William Cheng: The first step is now using a pace table when you try to access the memory location over here, we use the pace table, right. So the example that we saw before, is that

79
00:10:31.110 --> 00:10:36.180
William Cheng: You try to access. One of the memory locations over here, right. So in this case, we're going to look at whether you have access or not.

80
00:10:36.480 --> 00:10:43.110
William Cheng: If you have the right kind of access, then you are allowed to use it and this case, you know the the the the the the

81
00:10:43.650 --> 00:10:49.320
William Cheng: The physical address over here will point to, you know, four kilobytes of pages over here and you access. One of the memory location, they're

82
00:10:49.650 --> 00:10:58.500
William Cheng: There so that will be great, but if you access the memory location over here it says no access, while in that case you trapped into the operating system and the system will try to resolve.

83
00:10:58.980 --> 00:11:09.030
William Cheng: Resolve your virtual resolve your virtual address right so again when I try to access memory location over here I'm using a virtual address over here and now the page table says there's no access

84
00:11:09.660 --> 00:11:16.050
William Cheng: Okay, so, so sort of the front line, you know, defense to say, well, you know, the you know the pace that says you have no access to this page.

85
00:11:16.260 --> 00:11:20.790
William Cheng: And now we need to traveling the ordinances that and the opportunities that will try to resolve this address

86
00:11:21.180 --> 00:11:33.750
William Cheng: Okay, is the obvious ism and try to sort of figure out, you know, so, so, so this case, the resolution, meaning that given this virtual address, does it really mapped to some kind of those are really mapped to, you know, some kind of a

87
00:11:35.130 --> 00:11:37.200
William Cheng: Does it really matter part of your address space.

88
00:11:38.130 --> 00:11:44.940
William Cheng: There. So again, the picture that we mentioned before, we hear part of your address space mapped real memory and part of the outer space map to the desk.

89
00:11:45.150 --> 00:11:53.250
William Cheng: Right. So, therefore, when you're traveling the organism over here. They need to see if the, you know, if the memory reference that you're making doesn't actually exist on the desk.

90
00:11:53.460 --> 00:11:58.950
William Cheng: Okay. If it doesn't, under this one in this case you access a legal memory location. So in that case it will kill your program.

91
00:11:59.220 --> 00:12:07.800
William Cheng: But if, but if it turns out it's on the desk. Then what we should do is that we should bring the data from the days and move into memory. So therefore, in this case we will resolve their memory reference

92
00:12:08.460 --> 00:12:16.530
William Cheng: Okay, so what we need to do is that we need to we need to go to that this copy that data and the allocated for kilobytes pages over here and then fix this point are we are 2.2

93
00:12:16.890 --> 00:12:27.480
William Cheng: And then change this one to, you know, rewrite a real only only whatever it is and then we need to go back into the user space program to say, hey, we have we have fixed it will have fix it for you already. So now, please try again.

94
00:12:28.470 --> 00:12:39.240
William Cheng: Okay. So this guy is what it will do is it, why should try to access the memory. The second time. So hopefully everything is fixed by the operating system. So when you tried it the second time it will work. And then in this case, you won't trap into the opposite.

95
00:12:40.260 --> 00:12:46.680
William Cheng: Okay, so this is what it means by the operating system will try to resolve their memory reference when you get a paste all and you drive into the office.

96
00:12:47.190 --> 00:12:57.240
William Cheng: Yeah, alright. So, so, in the event that particular paste fault cannot be resolved, then the conceptually was that is going to turn into a segmentation fall

97
00:12:57.810 --> 00:13:06.150
William Cheng: So we talked about, you know, the unique signal cause six seven v. Right. So in that case, you know, something bad happens at a kernel and what you would do is they will deliver the signal.

98
00:13:06.360 --> 00:13:09.450
William Cheng: If you have a signal handler, then it will call it otherwise gonna kill your program.

99
00:13:09.900 --> 00:13:22.620
William Cheng: Guys, a nutcase. We started out with a pace file you tried to access a particular page and you get you and you gotta, you tried to the operating system. So that's a pitfall that can get turned into a segmentation fall when the memory reference cannot be resolved.

100
00:13:23.460 --> 00:13:30.330
William Cheng: Okay, when the memory reference can be can be resolved was in that case is going to be a good pace fault because obviously that will be able to fix it.

101
00:13:31.380 --> 00:13:41.190
William Cheng: All right. Okay. So, so over here says more peaceful is not a segmentation fall if you know the if it can be resolved and then everything's gonna be okay. Yeah.

102
00:13:43.710 --> 00:13:50.400
William Cheng: All right, so, so, so, so then how does, how does this done right and the reality, the operating system is too slow.

103
00:13:50.670 --> 00:13:57.750
William Cheng: To to perform you know the the address translation, you know, every step. So happened is that some of the stuff needs to be done using hardware.

104
00:13:58.140 --> 00:14:04.920
William Cheng: In the good old days, you know, some of the harder words over here are given the complete map to my virtual address to physical addresses.

105
00:14:05.640 --> 00:14:16.950
William Cheng: Okay, so that's no longer look no longer the case. I mean, these days, you know, lots of the operating system basically they use the data structure like this called a page table and the entire page table is not given to hardware.

106
00:14:18.030 --> 00:14:27.660
William Cheng: Okay, so in the good old days, the entire page table is given to hardware. You can imagine that if this you know this page table is four megabytes. You need to give four megabytes to hardware and that's a lot of memory.

107
00:14:28.470 --> 00:14:34.620
William Cheng: Okay, so, so, so the reason that it might be doable, long time ago. Is that because the page day well might be a little smaller.

108
00:14:34.920 --> 00:14:40.950
William Cheng: In the good old days, because the CPU is actually smaller right today, you have to add you have virtual address space of four gigabytes.

109
00:14:41.160 --> 00:14:50.700
William Cheng: In this case, we're going to end up with four megabytes a page table. If you get out to the if you give that to the CPU is going to take a long time for the CPU to take the entire data structure and put inside of CPU.

110
00:14:51.660 --> 00:14:56.220
William Cheng: Okay, so, so today we only put a little bit of the page table into the CPU at a time.

111
00:14:57.300 --> 00:15:02.940
William Cheng: Okay, so, so over here says, you know, in other cases only a partial map is given to the hardware.

112
00:15:03.990 --> 00:15:11.280
William Cheng: Okay, so we're going to take the entire the entire page table up here. That's four megabytes long over here, but we're only going to give a few entries over here.

113
00:15:12.540 --> 00:15:18.180
William Cheng: To the hallway at a time. And so this way, you know, things will be more scalable now. Alright.

114
00:15:18.870 --> 00:15:24.120
William Cheng: So either case, it is the obvious isn't me to provide some map, you know, to the hardware, you know, and

115
00:15:24.630 --> 00:15:32.490
William Cheng: And also build a data structure for the map, as I mentioned before, the page table is a data Colonel data structure that's understood by the hardware.

116
00:15:33.360 --> 00:15:38.970
William Cheng: OK, so the page that will be as part of the virtual memory map right because because the again. What is the purpose of the page table.

117
00:15:39.210 --> 00:15:49.680
William Cheng: The page table map virtual address the physical address. So that's kind of a mapping. Right. So we also call it an address translation that would take one address and cover into the other address that. So again, this is no as

118
00:15:50.520 --> 00:15:51.570
William Cheng: It gets up, I've taken a

119
00:15:52.140 --> 00:16:03.600
William Cheng: Data structure class. This is known as some sort of a map that given you know some some some some key you try to look up a value over here. So this case, the key will be the virtual address and then the value will be healed the physical address

120
00:16:04.260 --> 00:16:08.910
William Cheng: OK. So again, all these things are sort of been you know the different ways to sort of talk about the same thing that

121
00:16:09.420 --> 00:16:16.320
William Cheng: And then you know the inside of the Colonel, we have the virtual memory there a map data structure. I think we next is actually known as VM map.

122
00:16:16.800 --> 00:16:25.830
William Cheng: So VM again VM sense for virtual memory and this is the map data structure and the goal here is to map a virtual address. Eventually, the physical address. OK.

123
00:16:26.910 --> 00:16:33.810
William Cheng: So again, it's complicated because some of the information are stored inside the pace table will be here so that they can be fast.

124
00:16:34.050 --> 00:16:40.950
William Cheng: Okay, but, you know, in case you don't have enough physical memory. Well, then in that case some of the information has to be stored inside the colonel data structure in a

125
00:16:41.310 --> 00:16:45.300
William Cheng: Story in a more extensive data structure that we call the virtual memory map. Okay.

126
00:16:45.720 --> 00:16:53.550
William Cheng: So, so, so, so, so this is this way when you get a pace file inside the Colonel, we can actually look up that data structure and figure out how to fix your patron.

127
00:16:54.090 --> 00:17:02.490
William Cheng: Guys, when we, when you paste hard his face, then it doesn't turn into a segmentation fault, then so so again chapter sevens and talk about the detail.

128
00:17:03.090 --> 00:17:09.930
William Cheng: As it turns out, the virtual memory map is also how you what you use to implement the address space because because, you know, what is the address space.

129
00:17:10.950 --> 00:17:20.040
William Cheng: I mean, if you think about what is the address spaces address space is your, you know, it's basically it's a range of your virtual address when you sort of figure out which part of the map to

130
00:17:20.670 --> 00:17:24.930
William Cheng: Map. The physical memory. And which part of that I'm after this, and that is the virtual memory map.

131
00:17:25.950 --> 00:17:35.760
William Cheng: Okay, so therefore your, your address space representation and the virtual memory map. They are exactly the same thing. Okay, we're gonna use the virtual memory map to implement your address space.

132
00:17:37.350 --> 00:17:47.040
William Cheng: So, by the way, we also mentioned before, you know, inside your four gigabytes of address space. The first three gigabyte belong to the user. And then the last gigabyte belong to the tip on to the Colonel.

133
00:17:47.400 --> 00:17:54.990
William Cheng: Okay, only the users only the user portion of the address space need a virtual memory map. Okay. The rest of them over here.

134
00:17:55.410 --> 00:18:04.650
William Cheng: Belong to the Colonel. The Colonel is very powerful. The Colonel knows everything. So the colonel doesn't really need a data structure to represent the address space. Okay. The Colonel knows what the colonel look like

135
00:18:05.970 --> 00:18:12.180
William Cheng: Okay, so what the colonel needs to do is effort for every user space program. It needs to build a virtual memory map so they can manage

136
00:18:12.630 --> 00:18:20.250
William Cheng: The address space for each and every user space program. Okay, so for the last goodbye over here. The Colonel doesn't really need a data structure now.

137
00:18:21.150 --> 00:18:31.980
William Cheng: Right, so, so the virtual memory map and also what you would you can you what you wish for us to implement the memory map out where you map a file into your address space, you add that to your virtual memory map.

138
00:18:32.280 --> 00:18:40.320
William Cheng: Okay, so, so he said your virtual memory map, you're going to keep track of all your memory segments of again. What are your memories segment the tech segment that data segment of BSS the heat.

139
00:18:40.530 --> 00:18:47.520
William Cheng: And if you map a file into your address space, you're going to get a brand new memory segment. So again, you keep track of that inside your into your virtual memory map.

140
00:18:47.790 --> 00:18:53.190
William Cheng: Okay, and also the last one over here is going to be a stack segment. So all that information is kept track of inside the virtual memory that

141
00:18:54.300 --> 00:19:04.440
William Cheng: Now, so over here. Here's a picture of what a high level view of our virtual memory map looks like right we're going to start it all inside of process control blah inside process on your blog we saw before.

142
00:19:05.010 --> 00:19:12.570
William Cheng: You know, one of the, you know, one of the field. Is that a process control blog is your address space representation, right. So now we know what address space representation is

143
00:19:12.750 --> 00:19:17.790
William Cheng: The address space representation for the user space address is simply a virtual memory map.

144
00:19:18.120 --> 00:19:25.170
William Cheng: Okay, so they have over here with that address space representation. Now, can actually write the Ma ma P over here. So that will be the data structure.

145
00:19:25.800 --> 00:19:33.750
William Cheng: That they implement the virtual memory map there. And what is the virtual memory map the virtual memory map. It's a linked list of memory segments.

146
00:19:34.950 --> 00:19:48.000
William Cheng: Okay, so in this case over here in the textbook, they call these memories when they call it address space regions as regions as things were address space. So what does what. So what is the address space region address space region is a memory segment.

147
00:19:48.990 --> 00:19:59.070
William Cheng: There's over years. Okay, they're contiguous virtual addresses that have the same kind of stuff in that. Right. So the first one over here is going to be the tech segment right because all everything here, over here are your coat

148
00:19:59.340 --> 00:20:12.690
William Cheng: Rack. So, so what makes them so what what makes them similar they are all readable and they're all executable right over here. Let me on that. Right, so they're both readable and they're executable. And also we mentioned before they are using share mapping

149
00:20:13.770 --> 00:20:20.460
William Cheng: Okay, so therefore we're gonna have this data structure over here that keep track of what the memory segment look like. So again in as region. What do you need to know.

150
00:20:20.760 --> 00:20:28.320
William Cheng: Right. So number one is that for every memories that we need to know the first virtual address. We also need to know the last virtual drive so that we know that you know in this contiguous

151
00:20:28.560 --> 00:20:32.220
William Cheng: Virtual Memory range that you know that that is a memory segment.

152
00:20:32.700 --> 00:20:38.010
William Cheng: Okay, so in this example over here with that this as regional view go from 1000 and this has

153
00:20:38.280 --> 00:20:45.720
William Cheng: All the way to seven F F F. Right. So again, the typical way that we're going to write it is going to be zero, x 1000 comma

154
00:20:45.960 --> 00:20:53.220
William Cheng: Zero X 8000 again what we write things like that. The left hand side over here, we use a square bracket and the right hand side over here, we use

155
00:20:53.850 --> 00:21:01.860
William Cheng: You know, open bracket right to say that this particular range over here include the beginning but it doesn't include the, the last memory location so therefore he ends right here.

156
00:21:02.640 --> 00:21:08.460
William Cheng: Okay, so you can read it either memory location of either notation over here. I mean this notation over here we're going to end up with a little more character.

157
00:21:08.820 --> 00:21:16.680
William Cheng: So we can try to save some space. So therefore, but he's a dash and then mission. The final memory location. Now, so this, this is the text. I've been over here.

158
00:21:17.220 --> 00:21:23.220
William Cheng: So we need to know the start address. We need to know the length of the memory segment or the final location.

159
00:21:23.580 --> 00:21:30.300
William Cheng: That the final virtual address for that memory segment either either interchangeable. Yeah. And then we need to know what the access rights are, you know,

160
00:21:31.080 --> 00:21:38.940
William Cheng: What is the access. Right. You know, for this particular memory segment. So for the tech segment. They are read only. And they also they also need to be executable.

161
00:21:39.510 --> 00:21:45.060
William Cheng: Because what does that mean that memory. Memory second executable. That means that you can point he IP to it.

162
00:21:45.960 --> 00:21:56.130
William Cheng: Right, because if you have VIP VIP point is this one. Why would you point your IP to it because you're going to execute code from it. Okay, so when something's executable. That means that VIP is allowed to point to it.

163
00:21:57.180 --> 00:22:01.260
William Cheng: Okay. Otherwise, you're gonna get a segmentation fall and then I'll do the Colonel's gonna tell you

164
00:22:01.800 --> 00:22:08.850
William Cheng: Now, and finally over here, we need to remember every memory segment, whether they are shirt or their private so the text. I went over here. We know their shirt.

165
00:22:09.660 --> 00:22:18.870
William Cheng: Now, so over here is is a vision over here, the data sugar content, these four things right start address the length of the memory segment over here. So I guess to start as restore virtual address

166
00:22:19.140 --> 00:22:29.040
William Cheng: The access permission for this memory segment. And then finally, whether it's share a private okay if okay so also if this you know this as region or there's the

167
00:22:30.000 --> 00:22:36.120
William Cheng: Memory segment maps to a file. Well, in that case, we need to have a point to point to the corresponding file object.

168
00:22:37.050 --> 00:22:47.130
William Cheng: Okay, so in this case the text region, where does the Texas region come up. They come from executable file right you know because when you make the exact system call. We're going to go to the fall and then we're going to build the, you know,

169
00:22:47.940 --> 00:22:54.960
William Cheng: The tech segment from the executable file. Okay. So, therefore, this one will point to a file object. What would the file object do it for you.

170
00:22:55.380 --> 00:23:04.350
William Cheng: Guys walking is that will you need data from that particular file, you will ask the file after to take, could you get some data for me and this file object will know how to go to the desk and get data for you.

171
00:23:05.190 --> 00:23:07.170
William Cheng: Okay. So, therefore, you know, in the case that

172
00:23:07.590 --> 00:23:18.660
William Cheng: You know that the that you need physical memory for this particular memory segment. Whereas, in that case what you need to do is that you need to ask the buddy system for four kilobyte pages, right.

173
00:23:18.840 --> 00:23:28.500
William Cheng: And then you ask the fall object over here to go to the dish trust or four kilobytes from the dis into memory over here and then you change your memory map so that the application pullback and it uses memory.

174
00:23:29.760 --> 00:23:38.910
William Cheng: OK. So again, you know, if a memory segment come from a file. Well, in that case, you will point to a file object and fall object will be able to go to the disk and get data for you.

175
00:23:39.780 --> 00:23:50.610
William Cheng: Know, so the second one over here as well. This is, this is, you know, data plus BSL segment over here, right. So again, you know, they belong to the same memory segment because they have the same characteristic

176
00:23:51.120 --> 00:23:57.840
William Cheng: They are both rewrite and also their private. Right. So again, they are label private because we need to do copy on write on them.

177
00:23:58.260 --> 00:24:02.430
William Cheng: Right. So in this case over here. They also point to a fall over here. So when you try to read

178
00:24:03.180 --> 00:24:07.740
William Cheng: Read the data. Okay, what you do is that you ask the file object to transfer data from the data into memory.

179
00:24:07.950 --> 00:24:15.660
William Cheng: You change it, you know, the same way as we mentioned before, and then you set up the memory map over here and now you put on your user. But what happened when you write it for the first time.

180
00:24:16.020 --> 00:24:21.480
William Cheng: Will you write it for the first time since, you know, so, so this is this pretty good memory segment is private.

181
00:24:21.720 --> 00:24:30.780
William Cheng: That means that we need you to copyright you trapped into the operating system, the operating system is going to ask the buddy system for another four kilobytes. You make a copy of it. And now you right to the coffee.

182
00:24:31.470 --> 00:24:39.570
William Cheng: Okay, so why don't you perform copy on write for the first time. From this point on, you will always right into the copy. So therefore, you're not going to get the same pace fall for this particular address

183
00:24:41.100 --> 00:24:51.510
William Cheng: OK. So again, the know the the information is stored inside the as region over here, whether its private or sure it's very important because, you know, if it's private. That means that you have to perform copy. All right.

184
00:24:51.810 --> 00:24:56.460
William Cheng: There. So in this example, we're also start address and an address or start addressing the length

185
00:24:57.330 --> 00:25:07.440
William Cheng: OK, the next segment over here is going to be the heap segment right so again if you caught Malik and free. Then we're going to create the the heat segment. If you don't call Malik I'm free will then do the others memory segment is out there.

186
00:25:08.580 --> 00:25:15.420
William Cheng: Okay, so, so, so there's no reason to create an extra segment. If we don't need it. So this one again is a start and location. And then at the end, location.

187
00:25:15.600 --> 00:25:25.830
William Cheng: So for the heat segment. They also we right and they are also privately map right so they're doing copyright. So in this case, this point over here at the bottom. Over here, they will not point to a file object.

188
00:25:26.190 --> 00:25:27.180
William Cheng: But guess what would the point you

189
00:25:27.660 --> 00:25:32.100
William Cheng: Might as well get every, you know, every one of these data show here has a pointer over here. The point of some kind of object.

190
00:25:32.250 --> 00:25:44.820
William Cheng: So for the data actually come from a file, you have a file object. So for this one, we mentioned before, you know, this memory for the heap segment and for the stack segment. They are anonymous memory. So in this case, they will point to an object called the anonymous

191
00:25:45.480 --> 00:25:52.080
William Cheng: Anonymous. Anonymous object over here. So in this case, you know what you mean. And the physical memory. So what you would do is he will ask

192
00:25:52.200 --> 00:26:01.470
William Cheng: Anonymous object for and what it will do that. It will call out the buddy system allocate a four kilobytes over here and now it doesn't really need to copy any data into her because you know it's created out of thin air.

193
00:26:01.830 --> 00:26:11.610
William Cheng: So in this case, what it will do is that it can zero, you know, all these four kilobytes are bye bye bye initialize the multi zero in some operating system. They actually they don't initialize a memory allocated this way.

194
00:26:12.120 --> 00:26:18.840
William Cheng: Guys that, again, this is sort of a, you know, sort of operating system feature, whether they initialize your heat memory or not. Yeah.

195
00:26:19.530 --> 00:26:29.160
William Cheng: So the next example over here. So actually if you count of memory location over here. This is actually a one gigabyte file over here. Okay, so again if you try to map a one gigabyte file into your, you know,

196
00:26:30.360 --> 00:26:36.900
William Cheng: Into address space, what this could be down when you open the file and then you call em map you can map a map of find to address space.

197
00:26:37.110 --> 00:26:44.640
William Cheng: When you do that, you're going to create a new memory segment over here like this. Right. So again, this is a baby structure. It's a linguist, so you don't have to do that right you create a new

198
00:26:44.820 --> 00:26:50.670
William Cheng: Object over here you insert into the limiters over here. And since this one comes from a one gigabyte file over here. So again,

199
00:26:50.850 --> 00:26:59.550
William Cheng: The pointer over here to point to the file. So if you want, you know, so, so when you try to access your address space. That will be the same thing as accessing the file. So in that case. Uh huh.

200
00:27:00.030 --> 00:27:05.070
William Cheng: How does that happen, right. So when you access your address space over here, you're going to get. You're going to get a pitfall.

201
00:27:05.250 --> 00:27:12.210
William Cheng: You trying to the ordinances and then you find this memory segment and then you go to the fall over here is a treasure hunt data for me. So again, we're going to ask

202
00:27:12.600 --> 00:27:23.370
William Cheng: Trust for data four kilobytes at a time and then we're going to change the memory map over here, return back into the user's Facebook and now the user space program has access to the data from the files that are mapped into address space.

203
00:27:24.540 --> 00:27:32.100
William Cheng: Or so. So using these contracts, we can actually, you know, make it so that accessing your address space. It's exactly the same thing as its assets.

204
00:27:32.940 --> 00:27:38.760
William Cheng: That. And finally, the last one over here is going to be the stocks like men right here's the beginning I, J, here's the end address and you know this case.

205
00:27:39.150 --> 00:27:51.600
William Cheng: It's all rewrite and it's private. So we're doing, we're doing copy on right so in this case it will also point to anonymous object over here because there's no file system path that support this particular the support at this particular

206
00:27:52.710 --> 00:28:04.470
William Cheng: You know memory segment. Yeah. All right, so this is related to your kernel three assignment where you need to create and manage the address space and the virtual memory. Laughs Okay, so put an author is gonna be a lot of fun.

207
00:28:04.890 --> 00:28:08.910
William Cheng: Because you actually, you get to create this data structure in the kernel. Yeah.

208
00:28:10.710 --> 00:28:21.030
William Cheng: Oh I guess there's some additional notes over here at Texas Tech segment data plus BSS the heat the stag, the memory map father a map reroute or supply. So, by the way, if you read the

209
00:28:22.110 --> 00:28:25.560
William Cheng: If you, if you read the man pages for the map system call

210
00:28:25.860 --> 00:28:40.230
William Cheng: You will see that there are different ways to map a foreign to address space you can map share you can map it private. You can map it, we only have to rewrite. So in this example, we're mapping a rewrite and share. Okay. So also, you also there are other possibilities. Yeah.

211
00:28:43.320 --> 00:28:43.860
William Cheng: All right.

212
00:28:46.740 --> 00:28:53.310
William Cheng: Okay. So over here I guess I sort of mentioned some of this stuff already. So I'm going to skip this slide over here.

213
00:28:54.150 --> 00:29:07.260
William Cheng: Alright, so there are two issues that require further discussion. So one is, how is the primary storage manage in terms of resource management, right, because you know your physical memory is actually it's a precious resource.

214
00:29:07.680 --> 00:29:18.420
William Cheng: You don't have that much of it right if you think about your, you know, your laptop you're running 50 you know maybe 100 different application. Each one of them has a four gigabytes of address space. So that's what 200 gigabytes.

215
00:29:18.960 --> 00:29:32.760
William Cheng: You know that's 400 gigabytes over here and your physical memory is only you know for mega a four gigabytes a gigabyte 16 gigabyte. I mean, the dual days I guess in my office. I have a, I have a netbook that has only a one gigabyte of physical memory.

216
00:29:33.240 --> 00:29:45.030
William Cheng: So that people can also run 50 different processes. Okay. So, in that case, you know, how do I DVR this precious resource among all the all the application and the operating system that require memory.

217
00:29:45.570 --> 00:29:51.030
William Cheng: Guys over here. This is a resource allocation for our problem. So we're going to sort of briefly talk about how to do that. Yeah.

218
00:29:51.450 --> 00:29:54.780
William Cheng: All right, also, you know, the on the secondary storage, right, you have a desk.

219
00:29:55.020 --> 00:30:03.840
William Cheng: How do you manage objects on the desk right we mentioned, you know what, what kind of objects are you do you have on a desk right we have files and later on we'll talk about directories and all that kind of stuff.

220
00:30:04.140 --> 00:30:11.730
William Cheng: Okay, so all that stuff on the secondary storage. So this case, you know, how do you manage objects on the secondary storage. Right, right. So we're gonna come back to come to that. Pretty soon.

221
00:30:12.570 --> 00:30:21.510
William Cheng: Okay, you know, so. So first of all, you know, you know, who needs primary storage, right, all your application provide needs to upon me storage operating system, you know, these primary storage.

222
00:30:21.720 --> 00:30:29.280
William Cheng: Things I optimism. They're all kinds of different subsystem their device driver. They all need memory over here. So again, the memory is a precious resource.

223
00:30:29.730 --> 00:30:36.330
William Cheng: If the primary storage over here is poorly manage getting that case one subsystem can actually use up all the available memory.

224
00:30:36.810 --> 00:30:45.990
William Cheng: Okay, so for those of you who know about you, but there were security, it is possible that inside your networking substance them the attacker can actually sending sending a lot of

225
00:30:48.720 --> 00:30:59.250
William Cheng: Packets to you. So if you try to allocate too much, too much kernel memory inside the networking subsystem while pretty soon going to run out of kernel memory when you run on the current kernel memory, what do you do

226
00:31:00.690 --> 00:31:04.770
William Cheng: Okay, when your kernel memory. There's nothing you can do. But crashed the Colonel.

227
00:31:05.850 --> 00:31:11.250
William Cheng: Okay. So, therefore, you know, when you run out of kernel memory in the way that case is going to be a really serious problem. Right, so, so, so, yeah.

228
00:31:11.400 --> 00:31:22.260
William Cheng: We need to you know when when we tried to perform these resource management tried to allocate physical memory to all the the operating system components and the user space program that we need to be very, very careful.

229
00:31:22.620 --> 00:31:29.280
William Cheng: Like so. So again, it's this. This is the introductory class. We don't really talk about exactly how to do that, you know, for things like

230
00:31:29.610 --> 00:31:36.180
William Cheng: The networking subsystem, right, because that's, you know, that's a topic that would only cover too much in this class. Yeah, for user space program.

231
00:31:36.660 --> 00:31:42.330
William Cheng: As it turns out, one of the super simple way to do is that you can i just says what every user space program has a fixed amount of memory.

232
00:31:42.960 --> 00:31:51.000
William Cheng: Okay, so for example, if you're allocating you know four kilobytes pages at a time. You can say that every, you know, user process can only have 10 you can only have 10 pages of memory.

233
00:31:52.110 --> 00:31:53.040
William Cheng: Okay, is that a good idea.

234
00:31:54.000 --> 00:32:03.090
William Cheng: That so so again that there will be a simple allocation scheme, right, if you try to solve a resource allocation a problem. You can use a very, very simple solutions to say every process will get 10 pages of memory.

235
00:32:03.360 --> 00:32:07.200
William Cheng: If you, if you start doing that, you're going to end up with processes that performed really poorly.

236
00:32:08.100 --> 00:32:13.800
William Cheng: Okay, so that's really not a good way to go, so they don't get any Chapter seven was sort of talk talk about how this can be done. Yeah.

237
00:32:14.340 --> 00:32:22.740
William Cheng: All right. And also, you know, you know, over over here also asked question. Is it fair that is it fair for every user process to get 10 pages of memory.

238
00:32:23.370 --> 00:32:32.640
William Cheng: Okay, some people say, yeah, well everybody get the same amount of course that's fair. But, you know, again, you know, as it turns out, you know, fairness is actually very, very difficult to define. Okay, I guess, over here.

239
00:32:33.720 --> 00:32:38.850
William Cheng: Page of x over here the textbook. Talk about some more dense issues over here. We're going to skip that.

240
00:32:39.990 --> 00:32:45.690
William Cheng: I was also very difficult to fair To be fair, it's also very difficult to define for welfare means

241
00:32:46.530 --> 00:32:56.430
William Cheng: Okay, so we're used to, you know, when we give every program over here are, you know, sort of, sort of 10 pages. What is the program that you're giving 10 pages to that that program only needs five pages.

242
00:32:57.720 --> 00:33:02.460
William Cheng: Okay. So if that's the only five page. If you're gay. If you're giving them 10 pages is that unfair.

243
00:33:03.060 --> 00:33:07.590
William Cheng: Or something was so unfair because you're wasting share resources. So therefore, you're really not allowed to do that.

244
00:33:08.280 --> 00:33:15.570
William Cheng: OK. So again, you know, that's a fair. This is very, very difficult to defy So later on in chapter seven. We also going to look at, you know, some of the definition of fairness. Yeah.

245
00:33:16.440 --> 00:33:20.610
William Cheng: Alright, so, so for now we can sort of assume that we can actually use very, very simple solution.

246
00:33:21.060 --> 00:33:31.200
William Cheng: For every, you know, every process for every, you know, Colonel subsistence. We're going to sort of give them some kind of a minimal allocation. Okay, so this one is that, you know, whenever you create a process that guaranteed to have a minimum

247
00:33:32.430 --> 00:33:33.240
William Cheng: The minimum

248
00:33:34.530 --> 00:33:40.650
William Cheng: Amount of physical memory and whenever you have a substance and these are your insight, your kernel. You also allocate a fixed amount of memory.

249
00:33:41.310 --> 00:33:51.360
William Cheng: To them. So in that case, they allowed to use their private memory there allocate to that. But if they need more than that way that case, they have to compete with all the other processes, they have to compete with the operating system.

250
00:33:52.410 --> 00:33:58.050
William Cheng: Okay, so that would be a very simple model. Okay, so again, you know, in chapter seven will look at more details. Yeah.

251
00:33:59.790 --> 00:34:07.050
William Cheng: All right. How do you sort of manage objects in the secondary storage, right. So the solution over here is that we're going to build a file system.

252
00:34:07.500 --> 00:34:12.900
William Cheng: On top of the secondary storage device. Okay, so that means the first attempt to manage object in the second

253
00:34:13.290 --> 00:34:25.350
William Cheng: Story. So again, what's inside of our system right instead of us is that we have a file system hierarchy. We have directory of sub directory of sub sub directory and then we have files all over the place. We're going to build this data structure on your desk.

254
00:34:26.160 --> 00:34:36.180
William Cheng: Okay, so definitely so so so what is the file system. So unfortunately when we talk about us is that we actually we means two different things, right. Number one is that how do you lay out data on the secondary storage device.

255
00:34:37.080 --> 00:34:44.640
William Cheng: Okay, so in this case we are building data structure on the desk. So we talked about the file system, you could be talking about what is the data structure on this.

256
00:34:44.790 --> 00:34:48.840
William Cheng: So that this way you can build, you know, thought you could build files directory is all that kind of the kind of stuff.

257
00:34:49.530 --> 00:35:03.210
William Cheng: That the other thing is that you're, you need to write code to access the data. So therefore, how to access all those data in the data structure. So when you talk about the operating system code to access the file system. Well, you know, okay, that's also part of the houses them.

258
00:35:04.470 --> 00:35:11.130
William Cheng: Okay, so the second part over here, right, the toe that you wrote when he tried to access the file system. Don't you have to know what the data structure look like

259
00:35:12.390 --> 00:35:15.090
William Cheng: Okay, if it turns out there are different ways of layout data structure on

260
00:35:15.540 --> 00:35:21.570
William Cheng: You know, on the desk. Well, in that case, you're going to end up with different code because different code understand different kinds of data structure on desk.

261
00:35:21.960 --> 00:35:27.570
William Cheng: Okay. So, therefore, if you lay out data structure in a specific way on the desk. You have to use the code that come with it.

262
00:35:27.960 --> 00:35:33.870
William Cheng: Right, because otherwise, if you use somebody else's code then when it goes through this, they're going to be surprised because the data structure is not what they expect

263
00:35:34.410 --> 00:35:41.580
William Cheng: Okay, so what. So therefore, when you talk about the file system you're talking about two things, how the data is allocated on how the data is structured on the desk.

264
00:35:42.090 --> 00:35:48.660
William Cheng: Okay, so these are on this data structure and also you know the code that you need to use to access data structure on the desk. Yeah.

265
00:35:49.620 --> 00:35:57.390
William Cheng: Right. So typically, you know, so, so, so again, in that case it seems to be very, very specific if we want to access that this, we need to know how the data is

266
00:35:58.410 --> 00:36:04.200
William Cheng: How the data is allocated onto the desk. So in that case, it's going to make the job of the operating system programmer very, very difficult.

267
00:36:04.470 --> 00:36:13.560
William Cheng: Rabbit as the audiences and programmer needs to know what kind of file system they're dealing with. So this way, they will use different co Tu, Tu, Tu, Tu, Tu, Tu, Tu the access different parts of the desk.

268
00:36:14.190 --> 00:36:19.500
William Cheng: Okay, so again, that's not a good idea. We want the operating system to be file system independent

269
00:36:20.610 --> 00:36:28.950
William Cheng: Okay, so this way, no matter how you lay out data on the desk. Okay, the operating system is going to use the same code, right. So how do we do that now. So this is done.

270
00:36:29.700 --> 00:36:38.070
William Cheng: By dividing by dividing the file system into two different parts. One part is file system independent and the other one is file system dependent

271
00:36:38.430 --> 00:36:46.260
William Cheng: Okay, so, so the pop over here. So the problem is is the first is the independent par So we call this in the UNIX ism is called the virtual process and

272
00:36:46.800 --> 00:36:49.020
William Cheng: So that's the name of our Colonel Simon number two.

273
00:36:49.740 --> 00:37:01.050
William Cheng: Okay, so the top over over here is going to be the virtual file system over here, they support the file abstraction. Okay, so what does the fall abstraction. What we talked about that before a file is simply a real bytes.

274
00:37:01.590 --> 00:37:08.010
William Cheng: So if you think about a file is simply an array of bytes. Does it matter what data structure use to store them under the desk.

275
00:37:08.700 --> 00:37:13.620
William Cheng: While so no right and so it doesn't really matter how you think you can always, always, you know, think about the abstractions that

276
00:37:13.890 --> 00:37:23.160
William Cheng: Is simply a way of bytes. There's a file size. And if you want to make the font bigger you right beyond the current and if I become bigger, right, all these things we talked about in chapter one. That's the file extraction

277
00:37:23.910 --> 00:37:32.550
William Cheng: Okay, so for the virtual POS system over here, they're implementing the sort of the file system independent part. Okay. Because it doesn't really matter how you lay on the desk there.

278
00:37:33.360 --> 00:37:43.410
William Cheng: Alright so on Windows. They actually call this the Iowa management. Okay, so we have to do this later, we're still going to be a physical our management. So again, we don't really spend too much time talking about Windows

279
00:37:44.070 --> 00:37:53.310
William Cheng: The other part over here is going to be the file system dependent. So this one is called the actual file system on the Unix and Linux system right so VM FS the virtual bosses them their foster some independent

280
00:37:53.790 --> 00:37:58.320
William Cheng: So what kind of file system or the independent they're independent of the actual file system.

281
00:37:59.700 --> 00:38:09.270
William Cheng: OK. So again, the virtual file system over here cuz you know underneath it. We can have a FS number one FS number two. Number two, again, many, many actual bosses, then

282
00:38:09.780 --> 00:38:17.580
William Cheng: The virtual file system can work with any one of them there. So, so the actual versus them for for the Linux system or what are the Linux, you know,

283
00:38:18.030 --> 00:38:28.440
William Cheng: Actual classes that right if you have, you know, do any kind of an administration for Linux, you know that there's the St. Louis st 364. So these are the Linux file system.

284
00:38:28.860 --> 00:38:35.130
William Cheng: Right. So again, when you talk about UX team to you're talking about a very specific way of how to lay out data on the desk.

285
00:38:35.490 --> 00:38:43.020
William Cheng: Okay, you're also talking about the up the other the file system code that you will use to access the yes teacher file system on the desk.

286
00:38:43.770 --> 00:38:55.200
William Cheng: Okay, when you go to the st three that data structure on the days will be different. So therefore you need a different piece of code that will go with that for you to access data on the desk right yes the four is the newer generation of UFC three. So, yeah.

287
00:38:55.440 --> 00:39:05.850
William Cheng: The data on this is completely different data structure is completely different. So again, you need different coasts to go with it. Okay, the virtual versus them is going to be done in a way to independent of any of the actual file system.

288
00:39:07.530 --> 00:39:12.300
William Cheng: For Windows. Windows like to use generic names right so windows. The actual file system. They call it the file system.

289
00:39:12.990 --> 00:39:26.880
William Cheng: Okay. So, so what are the Windows file system. I have faith 1632 and PFS. These are all sort of the well known Windows file systems now. Right. So again, when they're talking about the process that you're talking about the actual file system. Okay.

290
00:39:29.220 --> 00:39:35.100
William Cheng: Alright, so let's go back to this picture before I said before, you know, before we have the page on the left over here, right. We said that, you know,

291
00:39:35.340 --> 00:39:40.320
William Cheng: You know, for every process. There's a file descriptor table. I see. Here's the PR process file descriptor table.

292
00:39:40.770 --> 00:39:44.400
William Cheng: Every file descriptor table entry is just a pointer to a file object.

293
00:39:44.580 --> 00:39:53.430
William Cheng: The Fall object is part of the system file table over here. We can also call it the file object table because they have an array of objects or a reality, they probably not implement as array.

294
00:39:53.610 --> 00:40:01.440
William Cheng: They might be implemented a linked list or some other fancier data structure. Okay, so there's a collection of fall object over here every fall off here has four fields. Right. We talked about that before.

295
00:40:01.710 --> 00:40:14.400
William Cheng: There's a reference com there is how you open the file to open a read only do you open a JIRA there's a cursor position. And then finally, there's something called I know pointer. I mentioned before, I know is how Unix and Linux used to represent a file.

296
00:40:15.660 --> 00:40:20.880
William Cheng: Okay, so again, where is the representation for the, you know, for the file. What they are in the actual file system.

297
00:40:21.780 --> 00:40:28.890
William Cheng: Okay, so therefore we hear this. I know point over here so we can actually draw a line over here on the left hand side over here is your virtual file system.

298
00:40:29.460 --> 00:40:39.930
William Cheng: Okay. On the right hand side, what do you have right so every idle part of your point to. I know. And this I know is I know it depends on the fossa, because every facet of them has a different representation of the I know

299
00:40:40.350 --> 00:40:45.930
William Cheng: The guys on the right side over here, this is the file system dependent parts on the right hand side is going to be the actual file system over here.

300
00:40:46.110 --> 00:40:58.170
William Cheng: Right, so we can sort of imagine that, you know, on the right hand side over here. There's going to be. I know table where every I know represent a file. So the idol plant over here. What we do what we do what the I know part over here was simply point to an entry is that I know table.

301
00:41:00.780 --> 00:41:08.250
William Cheng: Okay, so again, every actual file system will have a different kinds of Iowa table. Some of them might be bigger, something they might be smaller data storage is completely different.

302
00:41:08.490 --> 00:41:16.650
William Cheng: They need to map, you know, to the data structure on desk right and they belong to the actual file system that so this picture over here on you know when I draw the line over here.

303
00:41:17.430 --> 00:41:22.770
William Cheng: You know, so the file object and the I know it's on the left hand side, where the file object on the right hand side we have the I know

304
00:41:22.950 --> 00:41:30.480
William Cheng: They formed a boundary between VM Fs, and the FS on the left side we have DFS right so all these you know file descriptor table.

305
00:41:30.720 --> 00:41:37.020
William Cheng: All these foul logic over here. They belong to the virtual file system. And then on the right hand side over here. They are depends on the file system.

306
00:41:37.560 --> 00:41:42.300
William Cheng: Okay. They pointed the thoughts dependent stuff over here. Okay, so, so the question is, how can this be done.

307
00:41:43.230 --> 00:41:50.130
William Cheng: That so so the idea here is that, you know, the I know pointer over here can point to any actual file system. I mean, you know,

308
00:41:50.940 --> 00:42:02.550
William Cheng: So, so, so, you know, we know that we've been to 16.04 you can actually point to the Windows file system. Right. You have a shared folder, the shared folder is part of the Windows file system over here. There's the window system has. I know.

309
00:42:03.720 --> 00:42:12.690
William Cheng: Well, no, they actually don't. Right. I know is a Unix, Linux thing. So, so in that case is completely fake over here so you can actually point to a file system that doesn't even have I notes.

310
00:42:13.950 --> 00:42:26.820
William Cheng: Or so, therefore, again, on the left hand side it's going to be, you know, the virtual file system they're completely independent of the actual process and they can even, you know, you can even use this point a to point to actually have houses and that doesn't have. I know implementation.

311
00:42:28.470 --> 00:42:37.410
William Cheng: OK. So again, over here, you know, this point over here is going to be a you know like we saw before. In the divisor every going to be a polymorphous pointer there. So therefore, the implementation is going to be like this.

312
00:42:38.100 --> 00:42:44.700
William Cheng: That when we have a file object object has four things in here. There's a reference column. There's the access. There's a cursor position.

313
00:42:44.910 --> 00:42:48.570
William Cheng: And the rest over here is going to be one point there. So again, if you write your code in C plus plus.

314
00:42:48.780 --> 00:42:57.570
William Cheng: All the rest over here are virtual functions over here that do not have any implementation. So will you try to create the sub classes over here, right. So what are the salt passes are the file object.

315
00:42:58.560 --> 00:43:04.230
William Cheng: Okay, so. So in this case, again, if you are EFT to you're going to have an e x t to fall object.

316
00:43:05.250 --> 00:43:14.580
William Cheng: Okay, if you are an empty Fs, you know, File System object over here, you're gonna have an anti anti FS for file object over here. If you're a fast 30 to five, you know,

317
00:43:14.910 --> 00:43:20.970
William Cheng: Sort of fall over here. You gotta have a fit 32 file object. They're all sub classes on the file object.

318
00:43:21.510 --> 00:43:26.070
William Cheng: Okay, so therefore they all get reference kind of access or we can have a position over here, but they have to

319
00:43:26.340 --> 00:43:38.760
William Cheng: Re implement all these functions over here. So this way you know when you will, we will you use a 530 to fall object to create to create a file, there's a, how do you create a file on the fat 32 file system.

320
00:43:39.780 --> 00:43:46.260
William Cheng: Once again, the virtual file system doesn't know how to do that. Right. So, therefore, you have to use the actual file system. The act of us has them is going to be a fast 32 followed. Yeah.

321
00:43:46.500 --> 00:43:51.480
William Cheng: Of course a faster processor know how to create a file inside the fast 32 file system.

322
00:43:52.230 --> 00:43:57.600
William Cheng: Okay, but if you use it. Yes, T TO FALL object over here. We doesn't really know how to create a file object.

323
00:43:57.900 --> 00:44:05.850
William Cheng: So inside of your factory to file system. So again, you have to use the right one. So using sub classes over here, you can actually have all the implementation at the right places.

324
00:44:06.030 --> 00:44:13.920
William Cheng: And again c++. This is no as polymorphous them when we try to implement c++ and polymorphous me. See, we're going to end up with an array of function pointers.

325
00:44:14.370 --> 00:44:18.480
William Cheng: That. So therefore, in this case, what we end up doing is over here. Again, we're going to have a file object.

326
00:44:18.720 --> 00:44:25.500
William Cheng: This will be a data structure and see, well, so he's going to have the reference code is going to have the access to. It's going to have the cursor position.

327
00:44:25.740 --> 00:44:29.130
William Cheng: And then over here we can actually have an array of function pointers over here.

328
00:44:29.460 --> 00:44:38.070
William Cheng: Guys. Oh, yeah. Once your point is over here, you can have punches a pointer and Pointers. Pointers. Pointers or whatever. So this could be an array of function pointer. This could be also a pointer.

329
00:44:38.280 --> 00:44:43.620
William Cheng: To an array of function pointer or it could be appointed to a pointer to fit, but it does the detail, doesn't really matter.

330
00:44:44.160 --> 00:44:56.220
William Cheng: Okay, what's important to understand over here is that usually the fall, I get over here if you follow the pointer over here. You're going to point to the actual file system implementation of a specific operation for accessing the data on the desk.

331
00:44:58.620 --> 00:45:11.040
William Cheng: Okay, so again we're going to use a, you know, there's sort of the CBC implementation of the c++ polymorphous them. So this way you know your virtual assistant can be, you know, actual processing independent. Yeah.

332
00:45:12.630 --> 00:45:20.370
William Cheng: All right. And also, eventually, are you have to go to the hard drive over here. The hard drive over year that we're going to India when you try to get data from the desk over here.

333
00:45:20.520 --> 00:45:29.160
William Cheng: We're going to go through the device driver. So again, the operating system is not only device independent. It also you're at this level here, it needs to be actual process of independent

334
00:45:29.640 --> 00:45:39.810
William Cheng: OK. So again, the file system independent and device independent, they're actually different things there. But the ordinances and the Colonel is implement so that they can be independent of all this kind of stuff. Yeah.

335
00:45:42.300 --> 00:45:43.020
William Cheng: So,

336
00:45:44.880 --> 00:45:50.700
William Cheng: I said the last slide over here. So instead of process them, you know, I guess we're sort of have an introduction to the fastest them over here.

337
00:45:51.330 --> 00:45:57.330
William Cheng: You know the storage devices that they are really slow right your hard drive. So later on in Chapter says, want to see why is the hard drive so slow.

338
00:45:57.960 --> 00:46:05.610
William Cheng: So, so, you know, so there are different kind of this. There's a USB stick there. A little faster than the hard drive the hard drive. I actually super, super slow

339
00:46:05.940 --> 00:46:15.030
William Cheng: So therefore inside the colonel instead of houses in the implementation, they could get really, really complicated because we need to use a lot of tricks to make that this look like they are pretty fast.

340
00:46:15.690 --> 00:46:19.920
William Cheng: Okay, we're going to make them look and feel they're pretty fast, but in reality they're really slow devices.

341
00:46:20.250 --> 00:46:24.600
William Cheng: That. So one of the tricks that we can use something called a file system cache. And so you know what

342
00:46:24.990 --> 00:46:34.170
William Cheng: What is the cash right the cash is a look up data structure. Okay, so I'm going to tell you know what is that whenever you need to go to this first you need to check this data structure called the file system cache.

343
00:46:34.350 --> 00:46:39.630
William Cheng: To see if you have the data already inside of our system cache. If you have a data instead of kind of file system cache, then you can just use it.

344
00:46:40.050 --> 00:46:47.070
William Cheng: In this case, you don't have to go to the this so this way that this will look like. It's very, very fast because you actually found a data inside file system cache.

345
00:46:47.520 --> 00:46:54.060
William Cheng: And so, by the way, whereas, you know, where do you actually implement a file system cache. Well, you have to implement inside the inside the Colonel.

346
00:46:55.710 --> 00:47:07.620
William Cheng: Colonel inside of our system. So this guy's you're going to actually allocate physical memory to implement your file substance them so right so you can have a file system cache. OK. So the reason you want it. You want to do this is that

347
00:47:09.330 --> 00:47:14.670
William Cheng: Is that if you do this, then you your storage system will look like they're very fast.

348
00:47:15.540 --> 00:47:21.540
William Cheng: Now, so, so, so, so the data that you often use typically you will be able to find them inside of our system cash.

349
00:47:21.720 --> 00:47:27.240
William Cheng: If it turns out that data that you're looking for is not inside of our system cache. Why, in that case, you have to actually go to the storage device.

350
00:47:27.420 --> 00:47:38.250
William Cheng: Way for a long time for the data to get transfer eventually when I get transfer, you're going to keep a copy of the data inside the file system cache. So next time. Will you ask for it is going to be sitting right there. So, therefore, you don't have to go to this.

351
00:47:39.420 --> 00:47:45.540
William Cheng: Okay, so in this case we need to put up this file system cache is going to be a fantasy data structure. As it turns out, inside the

352
00:47:46.380 --> 00:47:54.180
William Cheng: Inside awkwardnesses them, you know, I guess. Now you're doing Colonel assignment number one. What is the most fancy data structure that you have run into

353
00:47:54.780 --> 00:48:00.480
William Cheng: While so far you have only run into linked lists, right, you have a double anything circular list. That's the most best data structure that you're gonna you're gonna run into

354
00:48:00.720 --> 00:48:05.730
William Cheng: Inside the file system because the file system is so slow. We're going to see all kinds of fancy data structure and

355
00:48:06.210 --> 00:48:09.810
William Cheng: Including a tree data structure or maybe some, you know, sort of advanced

356
00:48:09.990 --> 00:48:16.770
William Cheng: Data structure. We're also going to see, you know, maybe a hash table right the highest level can be used to locate you know data instead of file system cache.

357
00:48:16.920 --> 00:48:26.340
William Cheng: So if you're given an item number I can actually use that I know number to look up, you know, look up data, is that a file system cache. If the data is there. I will use it. If not, then I have to go to the desk and wait for it.

358
00:48:26.970 --> 00:48:32.280
William Cheng: OK. So again, I'm going to talk about more of these details in chapter six. Yeah. Alright, so now

359
00:48:33.300 --> 00:48:41.340
William Cheng: You know about the actual file system. So the next thing that we're going to do is that you know in part three of this today's lecture, we're gonna we're going to go back to chapter one.

360
00:48:41.970 --> 00:48:48.570
William Cheng: So so so the big secret over here is that all the stuff we talked about in chapter one. They actually belong to the virtual file system.

361
00:48:48.960 --> 00:48:54.000
William Cheng: Right when we talk about the file abstraction referral just a real bias, you know, want to make it bigger you right to the end.

362
00:48:54.210 --> 00:49:03.990
William Cheng: All that kind of stuff is the fall abstraction. So basically we're talking about the virtual classes, then we're talking about your kernel assignment number two. Okay. All right. We'll be back in a little bit.