WEBVTT

1
00:00:01.410 --> 00:00:07.620
William Cheng: Welcome to lectures 16 so we're just done with the midterm exam and Colonel two

2
00:00:08.610 --> 00:00:13.980
William Cheng: Is due next Tuesday. If you have co from previous semester, don't look at them. Don't copy them best to get rid of it.

3
00:00:14.610 --> 00:00:22.320
William Cheng: Grading guidelines on what grade section B tests is BFS test. And that's the main test, you have to pass.

4
00:00:22.860 --> 00:00:33.630
William Cheng: And section see your testing your BFS code together with threats after you make a submission. Make sure you verify your kernel submission. And again, make sure you have a good Readme file.

5
00:00:35.070 --> 00:00:42.030
William Cheng: So, Colonel three is bigger than Colonel wanting to combine so you should start doing Colonel three as soon as you're done with Colonel too.

6
00:00:42.810 --> 00:00:58.920
William Cheng: So the goal of today's lecture is to give you everything that you need to finish Colonel three there. So the colonel three is divided into, you know, sort of, I guess, going to sort of show you what kind of phases that we have a kernel three sort of divided into five different phases.

7
00:01:00.330 --> 00:01:09.210
William Cheng: So what's very important we do Colonel three is that at the beginning. Don't say be m equals two, one. If you read the Phoenix documentation is FBA me go to y. So don't do that yet.

8
00:01:09.660 --> 00:01:15.270
William Cheng: To start, so as far as you go to one. So you're using the system file, file system, no longer using the emphasis them.

9
00:01:15.750 --> 00:01:25.650
William Cheng: Said, be able to zero. And that means that you're still inside, Colonel. Okay, get those three functions to work in P frame Darcy and then make sure that BFS tests pass perfectly

10
00:01:26.220 --> 00:01:36.330
William Cheng: You don't have to pass the favor FS test because the favorite FS test is only designed to work with Colonel to. Okay, so when you go the colonel three don't bother with the paper FS test. Okay.

11
00:01:37.620 --> 00:01:47.400
William Cheng: Alright, so after that. So, so don't say VM equal to one, until you have a fairly reliable file system because we're going to rely on the file system to bring in the data, bringing that the pages for this into memory.

12
00:01:47.910 --> 00:01:59.640
William Cheng: So if that process is not reliable, then the rest of your kernel three is pretty meaningless. Okay, so you got to make sure that this works pretty well before we, before you said to me was one. Okay.

13
00:02:01.020 --> 00:02:09.300
William Cheng: Alright so here is the timeline, my recommendation for Colonel three, feel free to shift that up if you can finish Colonel too early.

14
00:02:10.050 --> 00:02:19.320
William Cheng: So lots of people are able to do that. So, so again, you should, you know, try your best. Phase one is what I just mentioned pass BFS has. What if I said equal to zero and be

15
00:02:19.860 --> 00:02:27.570
William Cheng: As far as equal one, a VM equal to zero. So, over here, I said, you know, should get this done in two days. The only three function, you have to implement

16
00:02:28.290 --> 00:02:36.960
William Cheng: You know you can do it faster. But as it turns out of what people end up doing is that they still have bugs. Other Colonel to so they need to continue to try to fix bugs that Colonel to

17
00:02:37.590 --> 00:02:43.080
William Cheng: So we hear you going to end up taking, you know, two days or even more so. So again after your, you know,

18
00:02:44.070 --> 00:02:51.240
William Cheng: So, so, so, so you should start Colonel through as soon as possible. You need to spend a lot of time on Colonel three because Colonel three is very difficult.

19
00:02:51.660 --> 00:02:54.540
William Cheng: Yeah. Alright, so once you can get the FS to test

20
00:02:54.750 --> 00:03:07.890
William Cheng: The two passages that VM equals 217 is why you need to start running user space program. So first only run one user space program, which is the hello program and we all know a hello world look like right you know you you started a program

21
00:03:08.070 --> 00:03:15.540
William Cheng: You print hello world and you quit. That's all you do. Okay, sounds really easy, but this is really, really hard and Colonel three. Okay.

22
00:03:16.110 --> 00:03:21.660
William Cheng: So so so what you need to do is that you need to build the address space to run as user been hello program.

23
00:03:21.930 --> 00:03:33.390
William Cheng: Directly from any progress using a function called colonel. Exactly. So the idea here is that in in progress, you are inside the inner process inside the colonel when you call Colonel exactly you're

24
00:03:33.600 --> 00:03:39.150
William Cheng: You're in a process is not a colonel will become the the intake process in the user space.

25
00:03:40.080 --> 00:03:49.260
William Cheng: There. That's how you start the user Facebook when you start on inside the Colonel, you call Colonel exactly you go into the user space. And now you are the user space in that process.

26
00:03:49.770 --> 00:03:56.580
William Cheng: The first argument in the exact system costs that what program to run. So in this case, your user space in a process, run the hello program.

27
00:03:57.630 --> 00:04:09.600
William Cheng: Okay. Normally, is you shouldn't do that right but but again we were just trying to get the Kota Kota work. So in this case, your user space in the program. Turns out it's Hello you print out a whole whole message and then a while. But that has nothing to do

28
00:04:09.990 --> 00:04:19.950
William Cheng: Because exit and exit will come back into the kernel. So in that case, you're in a process will die inside the kernel. And then they will wake up the idol process and idle process will turn off the machine.

29
00:04:21.060 --> 00:04:29.880
William Cheng: Okay, so the only difference between this and the previous case that you need to go in, you're going to pop into the user space do one little thing. And then you come into the kernel and then you turn turn off the machine. Yeah.

30
00:04:30.600 --> 00:04:35.610
William Cheng: All right, this is not easy and it's a lot of work, because you have to get all these things to to to work.

31
00:04:35.850 --> 00:04:45.270
William Cheng: You need to load a program into memory right when you call Colonel exactly you're going to invoke the loader. The loader is going to go to the you know the the hello program on the desk.

32
00:04:45.510 --> 00:04:50.970
William Cheng: And then what it will do is it will build the address space. So you have to write all the code to build the address space that

33
00:04:51.720 --> 00:04:58.320
William Cheng: So, so you need to build it and manipulate the address space and then once you once you finish doing that, you're going to go into the user space.

34
00:04:58.650 --> 00:05:06.120
William Cheng: For the first time, and the first time you going to use the space you're going to be in the startup or routine. And the first thing that will happen. This is not a startup, which means that you're going to get a page fault.

35
00:05:06.810 --> 00:05:17.850
William Cheng: Okay, so we cover that in the last lecture. So, so, so, so we're because we're doing on demand paging. So therefore, the first thing that you have to do in the user space program is to take care of the baseball. Okay, you actually have to write

36
00:05:18.180 --> 00:05:24.900
William Cheng: Write your code to take care of the pace file in the text aggregating the data segmenting the sex admin all the stuff that I mentioned before that.

37
00:05:25.350 --> 00:05:29.040
William Cheng: So again, you need to write code to handle pays off and also in the

38
00:05:29.640 --> 00:05:35.130
William Cheng: Hello provide you need to write to the terminal. Right. So, therefore, in this case, you have to open the device, you have to write to the device.

39
00:05:35.610 --> 00:05:47.340
William Cheng: So, therefore, you have to get some system called to work, such as the right system call and the open system called to work. Alright, so the user space. Boy, you're not allowed to change the color. So what it will do that will call open, you will call right and then it will call exit.

40
00:05:48.360 --> 00:05:57.690
William Cheng: Yeah. And in order for the right system for the word, what we will, you will see is that you know the right assistant call since it's a system called. It's a thin wrapper around the track machine instruction.

41
00:05:58.350 --> 00:06:08.880
William Cheng: So when you made the racism call you don't come into the kernel. And the goal is to get to the kernel version of the right function. Right. What is the current version of the right budget. It's do right that you're dealing with in Colonel to

42
00:06:09.390 --> 00:06:20.700
William Cheng: Okay, so when you make the racism, call it the goal is to go to do right and then since you're talking to a device. Eventually, as you reach a special file, file, right. As it turns out, it's very difficult to get right to get into do right

43
00:06:21.360 --> 00:06:34.350
William Cheng: Okay, so what you need to do is that you need to copy the, the, the function argument in the racism call copy that into the kernel. This will require you to convert a user space virtual address to a kernel space virtual address

44
00:06:35.280 --> 00:06:40.710
William Cheng: Okay. I mean, I think, today I'm going to sort of talk a little bit about how to do that. Right, so, so, so, so you would know how to do that.

45
00:06:41.070 --> 00:06:49.800
William Cheng: So he's not a cardinal, they also named their variable in a very special specific way. So this way you can actually look at them and try to sort of figure out what kind of virtual addresses.

46
00:06:50.010 --> 00:07:00.540
William Cheng: If you see a variable name call you add our guy, chances are it's a user space virtual address that means that this address is less than zero XE 000000

47
00:07:00.870 --> 00:07:09.720
William Cheng: Okay. And then if you see something okay at Dr. What that means that it's a kernel space virtual dress. That means going to be greater than or equal to zero, x zero, x is 0000

48
00:07:10.800 --> 00:07:17.550
William Cheng: Okay, I think we need to be very consistent. I don't know if there are exceptions. Right. But again, you should look at the variable name and use them as a hint and

49
00:07:18.180 --> 00:07:30.630
William Cheng: See what you have to know. All right, so, so the mind. Come on over here is that even though the winnings document to say that you should run a spin in that don't do that yet, because that's extremely difficult. You want to go go there in phases.

50
00:07:31.110 --> 00:07:43.050
William Cheng: So in phase two. If you can get hello to to work that will be really great. Okay. So, after that what you should do in Phase three is that you need to pass all the section B test without user space. Shout Okay.

51
00:07:43.590 --> 00:07:49.050
William Cheng: So look at section bc way of command, you have to do to pass again use exactly the same way as running the hello program.

52
00:07:49.590 --> 00:07:53.400
William Cheng: Phase four is to run as been in it. Okay, so when it is this

53
00:07:54.120 --> 00:08:05.580
William Cheng: espina is the real user space in that process rather than this guy's again you'll colonel in the process becomes a user space in their process. As it turns out of the universe space user space in this process is very, very simple. All it does is

54
00:08:07.740 --> 00:08:12.030
William Cheng: It is that it will create child processes and the child processes is going to be the user space show

55
00:08:12.450 --> 00:08:19.290
William Cheng: Okay, so that's how you get the user space show the user space. I'll look just like the colonel shout, right, except that you run a user space, then

56
00:08:19.860 --> 00:08:29.640
William Cheng: So Section C of the grading guy is running the user space. So make sure all the clothes command work. And then finally, you need to get all the section detail has to work. You know, so, preferably

57
00:08:30.060 --> 00:08:33.390
William Cheng: You want to run those co from the, from the user space shop.

58
00:08:34.290 --> 00:08:44.010
William Cheng: Now, alright. So again, this time is very, very aggressive, you should start as early as possible, you should finish up as early as possible. And there are students, you know, there are plenty of student in the end.

59
00:08:44.370 --> 00:08:53.280
William Cheng: Will turn in Colonel three, you know, with the 10% extra credit. So just, just to let you know that this can be done if your team work really hard and start really early.

60
00:08:53.640 --> 00:09:01.830
William Cheng: There and also Colonel three is where everything comes together, it is important that you read the colonel three FAQ is almost impossible is almost, you know,

61
00:09:02.340 --> 00:09:09.060
William Cheng: Not very understandable at the beginning. But once you start working on it. It will then it will start making more and more sense.

62
00:09:09.810 --> 00:09:19.260
William Cheng: So it's important that you read the colonel through FAQ and understand pretty much all the lecture material covered so far in class, everything's going to come together and Colonel three. Yeah.

63
00:09:20.760 --> 00:09:23.520
William Cheng: All right. If you have questions, feel free to send me email.

64
00:09:24.120 --> 00:09:26.910
William Cheng: Alright, so let's go back to talk about, you know,

65
00:09:27.240 --> 00:09:36.990
William Cheng: Chapter Chapter seven over here. So last time we mentioned a special case in the next right where there's only one gigabytes of memory and what you would do is that you will map the colonel virtual address one for one

66
00:09:37.620 --> 00:09:47.550
William Cheng: to two, two to the physical memory, right. So this way, the kernel code access, you know, the corn can access every physical address, you know, in a very straightforward manner.

67
00:09:48.330 --> 00:09:56.640
William Cheng: Okay. So conceptually. That's what needs to be done in reality you know the the scheme of things that occur know for various reason will be a little different from that as

68
00:09:57.600 --> 00:10:04.860
William Cheng: Well, so again, in that case, it's important to understand that, you know, you know, for every physical memory over here.

69
00:10:05.160 --> 00:10:14.880
William Cheng: You know if this was a memory is used by the user space program, then in that case this physical memory will have to virtual addresses one in the user space and one inside, Colonel.

70
00:10:15.270 --> 00:10:28.170
William Cheng: OK. So again, these two virtual addresses the offset of the same it just the the virtual paste number will be different. Okay, but the physical beta page I movie will be exactly the same. Right. Whereas when you work on your kernel three again you need to keep this thing in mind. Okay.

71
00:10:29.610 --> 00:10:33.780
William Cheng: And the next thing I'll show you in the last lecture is to show you that you know so so

72
00:10:34.110 --> 00:10:42.570
William Cheng: So so so in this case every page tables you sort of think about the page table. The first three quarter of the page table is used by the user space address translation.

73
00:10:42.810 --> 00:10:47.340
William Cheng: And the bottom quarter of the page table is used by the colonel address translation that

74
00:10:47.700 --> 00:10:55.320
William Cheng: For all the current processes. So since once you're inside the Colonel. The Colonel is behaving like a one giant process, even though there are Colonel processes.

75
00:10:55.770 --> 00:11:01.680
William Cheng: That means that the the page table over here for all the colonel processes. They are all map exactly the same way.

76
00:11:02.430 --> 00:11:05.340
William Cheng: So I guess today. What sort of going to go into and sort of stresses point

77
00:11:05.700 --> 00:11:17.520
William Cheng: So that you can so you can understand that you know that that that that that that the page table over here for all the colonel processes. They are exactly the same. Right. The only thing that's different is the user portion of the page table. Okay.

78
00:11:19.320 --> 00:11:25.800
William Cheng: Alright, so I sort of draw this picture over here, I have to program right one is application. While one is application to both of them are user space program.

79
00:11:26.010 --> 00:11:34.650
William Cheng: But their address space. The first gigabytes user and the last week about here's the kernel for application. Number two, the first three gigabyte overhears user. And the last part over here is

80
00:11:34.890 --> 00:11:45.900
William Cheng: Is also the Colonel. So yeah, one thing that you need to remember is that when both of them coming inside the Colonel, you know, as far as the colonel concern. This really is just one giant process.

81
00:11:46.290 --> 00:11:49.770
William Cheng: Okay, you've been doing that all along since your current one. And Colonel to

82
00:11:50.580 --> 00:11:57.540
William Cheng: Okay. So, so what do you choose to do so to think about your current one. And Colonel to whenever you switch from one process to another process.

83
00:11:57.780 --> 00:12:09.780
William Cheng: Does the address of current progress occur Prague is the current process that this is a global variable. Does the address of corporate changes, simply because you move from one kernel process to another Colonel process.

84
00:12:10.680 --> 00:12:18.630
William Cheng: Yeah, the answer is of course they don't change. Right. What about the location of the buddy system does any of the virtual addresses that have Colonel change.

85
00:12:18.810 --> 00:12:25.800
William Cheng: The answer is no. So once you're inside the Colonel, no matter which processes are running out of current all all the virtual edges are exactly the same.

86
00:12:26.190 --> 00:12:30.900
William Cheng: Okay, the virtual addresses. The only difference for the user part of a process.

87
00:12:31.530 --> 00:12:37.020
William Cheng: Okay, when you're using the bottom part of the code of any process, they all map exactly the same.

88
00:12:37.230 --> 00:12:45.180
William Cheng: Right. So, therefore, again, the mapping is done using the colonel data structure and also using the pace table. So again, over here, it says, you know, in order for you to perform address translation.

89
00:12:45.390 --> 00:12:51.510
William Cheng: To convert virtual address the physical address over here, you're going to use the pace table, you can use the virtual memory map and inside of a stable.

90
00:12:51.660 --> 00:13:03.210
William Cheng: The first three quarter can be mapped differently from one process to the other one. But the bottom one quarter of the page table. They are map exactly the same. And that's why in Kona one and two. You don't have to manipulate the page table.

91
00:13:04.500 --> 00:13:12.750
William Cheng: Okay, so, so in quarter one and two. You start with a page table where the optimism is initialized. And once that's initialize all you do is to copy the page table.

92
00:13:13.380 --> 00:13:16.590
William Cheng: Okay, so in quarter one and two, you can actually share them as long as you don't delete them.

93
00:13:16.890 --> 00:13:19.470
William Cheng: Right, you can actually show them that won't go causing a problem.

94
00:13:19.650 --> 00:13:26.880
William Cheng: Where you go to court or three. Well, then each one of them is going to be a separate page table. You want to make sure you copy that and don't share them with some people is going to have Colonel three bug.

95
00:13:27.090 --> 00:13:30.810
William Cheng: They share them and this guy's when you delete them, then you're going to end up with memory corruption bug.

96
00:13:31.650 --> 00:13:38.280
William Cheng: OK. So again, you need to make sure that you copy them for the bottom one quarter of the pace table for every process. They're all exactly the same.

97
00:13:38.760 --> 00:13:44.010
William Cheng: Okay, so this way when you come into the kernel all the global variable they stay exactly where they are right.

98
00:13:44.760 --> 00:13:51.720
William Cheng: Right. So over here, that's what. So what I'm saying before, and in quarter one and two, you know, we don't have a user a portion of the way

99
00:13:52.080 --> 00:13:57.810
William Cheng: We don't have any user processing. So for every page table. The first three quarter. Always have equal to zero.

100
00:13:58.350 --> 00:14:06.840
William Cheng: Okay, so that's why you know well if you're inside the kernel when you access the user space virtual address, you're going to get a paper, a colonel page file and you end up with a kernel panic.

101
00:14:07.350 --> 00:14:15.750
William Cheng: Okay, so that's one of the common you know error and kind of one and two, you get when you get a pic cuddle baseball. If you look at the pace for virtual address sometimes equal to zero. So, how you build a small number.

102
00:14:16.200 --> 00:14:22.740
William Cheng: So in that case, you know, because the go to zero. That's why end of the colonel virtual the that's what that's what you end up with the colonel page for that.

103
00:14:24.150 --> 00:14:31.050
William Cheng: Alright, so how to set up the first three quarter the pace table for user process. Right. So the first quarter of the user positive over here.

104
00:14:31.290 --> 00:14:39.720
William Cheng: There are different for every user process because again the picture that I showed over here is that, you know, for user process. Number one over here. Here's a page that map to this page and physical memory.

105
00:14:39.900 --> 00:14:42.960
William Cheng: And here's another page and users process number to over here.

106
00:14:43.290 --> 00:14:49.410
William Cheng: So the map of different portion of the physical memory over here. Now again you need to remember inside the Colonel, I can use a

107
00:14:49.590 --> 00:14:57.510
William Cheng: You know, for, for, for every physical memory location over here since the physical memory location is only one gigabyte in size. I can, I can always use any of

108
00:14:58.110 --> 00:15:03.600
William Cheng: Always use a colonel address to access any memory location any memory location is a physical memory.

109
00:15:04.500 --> 00:15:13.080
William Cheng: Okay, so. So, this way, this page over here for for the blue process that's mapping the physical memory when I'm inside the Colonel, I can use a kernel virtual address to change it.

110
00:15:13.950 --> 00:15:26.250
William Cheng: Okay. Similarly, when the the football for for this particular page that's mapped to this physical memory when I'm inside the colonel my, I can also use the colonel virtual address to change data over here or to read data from these user processes.

111
00:15:27.150 --> 00:15:31.440
William Cheng: All right. And again, the reason is over here is that because I only have one gigabytes of physical memory. Okay.

112
00:15:31.560 --> 00:15:45.330
William Cheng: If I'm one. The one is wondering why the physical memory that case things will get a little more messy. Okay, so, so one gigabytes of physical memory is actually very, very straightforward because I can just map, you know, virtual Colonel virtual address the physical memory, one for one right

113
00:15:46.830 --> 00:15:56.640
William Cheng: To build the other the other the rest of the page table over here. I'm going to end up using the virtual memory map data structures will get what is the virtual memory map data structure that's your address space implementation.

114
00:15:56.850 --> 00:16:06.660
William Cheng: That's a linear list of as region using that data structure, you can decide how you want to. How do you want to fill out all the entries overhearing the first three quarter, you know, of the page table. Okay.

115
00:16:07.590 --> 00:16:14.850
William Cheng: So, okay, here's a note over, he has to say the virtual memory map is only needed to manage to manage the user portion of the address space.

116
00:16:15.060 --> 00:16:22.020
William Cheng: Because when you are inside the Colonel. The Colonel doesn't need a data structure to represent it address space. And why is that because it's Colonel knows everything.

117
00:16:22.620 --> 00:16:35.100
William Cheng: OK, so the colonel doesn't need special data structure to to represent the address space where inside the cardinal right only the user portion of the space we need specialized data structure to represent them. And yeah, for every user process, they're all different. Okay.

118
00:16:37.350 --> 00:16:45.960
William Cheng: All right, so I guess I just mentioned this, this book for every physical address that's allocated to a user process. It can have two virtual addresses.

119
00:16:46.170 --> 00:17:00.330
William Cheng: One for the colonel on one for the user process and the reason for the Colonel. Is that because there's one for one mapping from a kernel virtual address to to to a physical address. So, therefore, you know, if this physical address belong to a user process, it must have actually

120
00:17:01.890 --> 00:17:09.720
William Cheng: For any physical address it has occurred over to address that, if, if it's also mapped to a user process. And now we're going to end up with two virtual addresses now.

121
00:17:11.580 --> 00:17:17.220
William Cheng: Here's the, said I, you know, when you're inside the colonel over here. And, you know, in order for you to access that they have

122
00:17:17.580 --> 00:17:22.260
William Cheng: This memory inside physical memory which virtual address that you have to use inside the Colonel.

123
00:17:23.010 --> 00:17:28.140
William Cheng: OK, so now when I means that I have two choices, right, because I have to virtual address to access one physical memory location.

124
00:17:28.470 --> 00:17:36.750
William Cheng: Which virtual is I, should I use. Okay, the answer should always be that if you are inside the Colonel, you should use the kernel version of the virtual dress. Right. Why is that

125
00:17:36.990 --> 00:17:43.440
William Cheng: Because the user space virtual address can give you a paste Hall. Right. So if you get a patient instead of current all along, and in that case is going to crash, Colonel.

126
00:17:44.040 --> 00:17:53.610
William Cheng: Okay. So, therefore, you know, be careful with user space virtual dress because when you try to use this user user space virtual address. What are the corresponding page table entry over here, equal to zero.

127
00:17:54.090 --> 00:17:59.850
William Cheng: Well, then in that case, you're going to end up with, they will with the colonel paste all and then you're going to end up crashing.

128
00:18:01.200 --> 00:18:12.330
William Cheng: So, therefore, you can always use the kernel space virtual address over here because they're always safe to use when you're inside a carnal. Right, right. Keep this in mind when you're doing your Colonel the colonel three assignment.

129
00:18:13.800 --> 00:18:16.440
William Cheng: Oh, are you trying to the current out right. So again, the idea here.

130
00:18:16.830 --> 00:18:26.910
William Cheng: Is that a current or is that who trappings out a colonel right you have a user threat you you made a system call and then you try things out a colonel. So again, you're still the same process. You know, you're still the same, so

131
00:18:27.210 --> 00:18:33.900
William Cheng: You still belong to the same process. So, therefore, you have the same page table and you have exactly the same address space.

132
00:18:34.560 --> 00:18:40.500
William Cheng: They are so again, keep in mind. Right. Well, you try these other Colonel everything's still the same, same process in page tables, same

133
00:18:41.310 --> 00:18:50.550
William Cheng: You know, same process control blog, you know, same thread control blog all the time stuff there in x86 there's actually a bit inside of page table entry over here.

134
00:18:51.000 --> 00:18:57.780
William Cheng: For your the top three quarter of the page. The page table. They all have P to have the user base at what

135
00:18:58.200 --> 00:19:09.360
William Cheng: are so good at the pace that over here it's a little different from the picture that I that we draw, but when you get to your kernel, sweetie, you should look at the colonel three FAQ, and it will point you to the sad three side the x86

136
00:19:09.630 --> 00:19:15.870
William Cheng: Programmers manual and it will show you exactly what a page table entry look like for the x86 CPU now.

137
00:19:16.260 --> 00:19:23.160
William Cheng: One of the bit over here is called the user bit if the user base set equal to one, that means that that page table entry is for the user process.

138
00:19:23.820 --> 00:19:32.010
William Cheng: Okay, so when I say that, you know, the user process. The first three quarter over here are used by the user space process. And that's because I said the user been over here, equal to one.

139
00:19:33.090 --> 00:19:43.560
William Cheng: Okay, it is possible. You can actually use less memory by setting the user space user ID over here, equal to one for less amount of a stable entry and they have more energy means that the Colonel, but typically that's nuts not done.

140
00:19:44.040 --> 00:19:47.730
William Cheng: In Linux. Okay. But again, this can be done for the Intel CPU now.

141
00:19:48.000 --> 00:20:00.780
William Cheng: The bottom one quarter of the page table entry. They're all have this set to zero. Right. Well, instead of zero over here. So, so, you know, some people actually refer to the spirit is equal to one. So that's where super user. And that means that means is the Colonel.

142
00:20:01.890 --> 00:20:11.040
William Cheng: Okay. So this guy is you build what we say go to zero options there is only one beer. Some people call it the up some people call it the SP you stands for user and stands for the super user or the Colonel.

143
00:20:11.910 --> 00:20:17.580
William Cheng: Okay, so. So again, if the user is equal to zero, that means that you're in the supervisor, Mel. So again, you do inside the Colonel.

144
00:20:18.930 --> 00:20:30.750
William Cheng: The Colonel part of the page table or Matt identical only for all the processes over here. So if you only have one gigabytes or less a physical memory one says pages map, they will never change.

145
00:20:31.380 --> 00:20:38.520
William Cheng: OK. So again, the bottom call a proportion. Okay. Simple entry right the bottom. Yeah, but by the one quarter of the page, there will be here.

146
00:20:38.760 --> 00:20:48.090
William Cheng: Once they are set up right so what we do is that we've mapped one for one, you know, the colonel virtual address the physical memory. Is there any reason we might want to change, you know the the paper entry.

147
00:20:49.350 --> 00:20:55.140
William Cheng: What we have a one for one mapping. So once you once you're done mapping, then one for one. There's no reason to change it ever

148
00:20:55.770 --> 00:21:02.040
William Cheng: Okay, so your current one and two. That's why you never have to change the pace table because once they are set up once at the beginning.

149
00:21:02.280 --> 00:21:09.720
William Cheng: You never have to touch that. Okay. In Colonel three, you have to start touching them but again you don't touch the bottom three quarter, because the bottom three quarter all the

150
00:21:10.050 --> 00:21:15.870
William Cheng: The Colonel Goldberg rebel, they all stay exactly where they are. So all you have to do is to change the user portion of the page table.

151
00:21:16.560 --> 00:21:26.850
William Cheng: Okay, so that's what you need to manipulate your, your, in your current or three assignment that and remember our lessons, you're doing on demand paging at the beginning when you first create a cooker.

152
00:21:27.420 --> 00:21:30.180
William Cheng: will will will will you first run a program

153
00:21:30.390 --> 00:21:41.910
William Cheng: The first three quarter of the page table all have equal to zero, what the bottom part over here you copy from the parent process. So therefore, the good be exactly the same as parents, right. And then again, why is your copy it, you'd never touch it. Yeah.

154
00:21:44.250 --> 00:21:56.790
William Cheng: Alright so here is a homework assignment go for quizzes really not a real homework. So I'm highly recommend you to read this car. This is called, it's called PT in it. It shows you how the page table is initialized at the beginning of Phoenix.

155
00:21:57.300 --> 00:22:03.570
William Cheng: Okay, so this is, you know, this function is since that page table Darcy over here. So again, read a little bit to get us to

156
00:22:03.900 --> 00:22:08.010
William Cheng: Sort of get an idea of what happens over here. And then again, don't try to understand everything there.

157
00:22:08.490 --> 00:22:10.320
William Cheng: Because, you know, things can get really, really messy.

158
00:22:10.590 --> 00:22:18.030
William Cheng: That. So what you will see is that, you know, the colonel types and the data of the BSS thought is segment. They all start at one. I mean, there are two members. I'm going over here.

159
00:22:18.240 --> 00:22:28.170
William Cheng: The tech side went over your starts at 000000. So at the beginning of the the colonel virtual address your tech side man and then come to the data, plus the BSS segment.

160
00:22:28.530 --> 00:22:35.910
William Cheng: Okay, so all the global other the kind of global variable, they're all sitting right there, right. What about the rest of that right. The Colonel doesn't really have a heap.

161
00:22:37.200 --> 00:22:38.970
William Cheng: Okay, why doesn't the currently so he

162
00:22:39.300 --> 00:22:51.960
William Cheng: Was because he had used the buddy system to allocate memory, right. So, therefore, it doesn't really need to heat that so then comes the Colonel's page directory table. So again, inside what you were using the Intel CPU, the entire CPU is a multi level page table.

163
00:22:52.440 --> 00:22:56.430
William Cheng: The first one is called the pace directory table and the second one will call it the regular page table.

164
00:22:57.000 --> 00:23:04.770
William Cheng: Okay, so each one of them is four kilobytes in size. So the next data structure at the allocated inside PT. And then over here is going to be the pace directory table.

165
00:23:04.950 --> 00:23:11.520
William Cheng: And then if you read the data review read a cold very, very carefully, you will see that this data structure is actually eight kilobytes in size.

166
00:23:11.790 --> 00:23:19.020
William Cheng: But in the beginning part of chapter seven. We talked about this one, the state of Georgia over here is four kilobytes. So why is it actually eight kilobytes of data structure.

167
00:23:19.290 --> 00:23:33.270
William Cheng: That, as it turns out, the page directory table over here. The first four kilobytes over here is what the the address translation users use the first kilobytes at the first four kilobytes. The second four kilobytes over here. As it turns out, it was just Colonel data structure.

168
00:23:34.500 --> 00:23:44.130
William Cheng: Okay, you know, so instead of weenies can only call this entire data structure the page directory table, even though the only the first four kilobyte is the real paste directory table as far as the CPU is concerned.

169
00:23:44.370 --> 00:23:54.660
William Cheng: The rest of it. They're just Colonel data structure. Okay, so again if you try to understand how that portion of the other portion of that data structure is used. Again, it's very, very, very, very ugly code.

170
00:23:55.380 --> 00:24:09.750
William Cheng: Guys, okay you know don't spend too much time on that. Yeah. And then it will allocate all the colonel page table right because we have about monolithic Colonel, we know exactly how much memory. The Colonel can have. So therefore, we can actually pre allocate all the kernel space page table.

171
00:24:11.010 --> 00:24:13.740
William Cheng: Or so we can actually see that all the patient care of his them over here is gonna

172
00:24:14.430 --> 00:24:20.790
William Cheng: It's gonna be a good alligator. And remember that, you know, for all the page table, you know, for all the cut over here. They all have exactly the same content.

173
00:24:21.510 --> 00:24:31.170
William Cheng: Okay, so therefore the content where they're pointing to over here. They're all allocated. They're all pre allocated and they are initialized. So from this point on, they don't have to change anymore right for the carnal. Yeah.

174
00:24:32.340 --> 00:24:40.170
William Cheng: And then Robin, is that right after that the rest of the physical memory are given given to the buddy system. So now the buddy system can manage physical memory.

175
00:24:40.620 --> 00:24:50.100
William Cheng: Okay, so, so now at this point at the end. The PT and then the entire memory that's used by the Colonel, they're all accounted for right there allocated for the tech segment for the data segment for the BSS

176
00:24:50.610 --> 00:24:56.280
William Cheng: For the colonel page table and the rest of it is handled by the buddy system. Right. That's your entire kernel memory.

177
00:24:56.880 --> 00:24:58.680
William Cheng: Then, so this is how they are initialized.

178
00:24:59.010 --> 00:25:05.310
William Cheng: Okay, right. So you need to understand is that a corner over here. Again, just like the user processes. They can only use virtual address

179
00:25:05.490 --> 00:25:14.940
William Cheng: Even though it is the the the buddy system that manages the physical address space. But again, the buddy system is Conoco. So, therefore, if you want to manipulate data, you can only use Colonel virtual addresses.

180
00:25:17.220 --> 00:25:34.590
William Cheng: Right, there's a little note over here readings only has 256 megabytes of physical memory. So, so the memory go from 00000000 right the 256 megabytes. So at the last memory locations GOT A, B, C, F, F, F, F, F, F, F. Right. So there's no D. There's no easy. There's no f

181
00:25:35.250 --> 00:25:40.950
William Cheng: OK, so those are the only allow you know the ritual justice inside carnal. Yeah.

182
00:25:43.560 --> 00:25:50.610
William Cheng: Alright, so over here is how you convert a user space virtual address to a kernel space virtual address. So, in

183
00:25:50.790 --> 00:25:59.610
William Cheng: Phoenix when an application called read and the buffer address over here. Right, so read has your read over here has three forms your argument. The first one is the file descriptor.

184
00:26:00.090 --> 00:26:05.190
William Cheng: The second one is the address of a user space buffer. So here's the buffer right here. Here's buffer right here.

185
00:26:05.400 --> 00:26:14.610
William Cheng: He has a user space virtual address and then the number of bytes in the buffer. Right. So what are you trying to do is that is to try to ask the current all to fill this buffer with data up to the size of the buffer.

186
00:26:15.120 --> 00:26:21.900
William Cheng: Okay, so when you make the racism called when you come inside the corner over here, you know, the buffer address over here. I'm going to use my favorite user space virtual dress.

187
00:26:22.110 --> 00:26:34.320
William Cheng: This address is 012345678 so so it seems that you know that the lead. They are the first, you know, a character over here in the hex address. It's not C D or F. So that means that this is a user space virtual address

188
00:26:34.800 --> 00:26:41.850
William Cheng: Okay, so what we need to do is that we know is that a colonel, we need to ask that this to transfer data, you know, what do we need to transfer the data from

189
00:26:42.150 --> 00:26:48.870
William Cheng: What the data need to come from the desk. What does it need to go to write it needs to go to a different place inside the Colonel, because you know

190
00:26:49.050 --> 00:26:55.350
William Cheng: This buffer over here by the really small. If we ask that this to transfer one sector of data into this buffer, we're going to end up having buffer overflow.

191
00:26:56.640 --> 00:26:59.370
William Cheng: Okay, so, so therefore, again, this needs to be done in multiple step.

192
00:26:59.670 --> 00:27:07.230
William Cheng: Inside the current we're going to ask that this to transfer data from this into some other kernel memory. And then we're going to copy, just the right number of bytes into this buffer.

193
00:27:07.560 --> 00:27:14.730
William Cheng: OK. So again, what we need to copy the link to this buffer what virtual address you, we use you know this, you know, so this one not to a physical memory location over here.

194
00:27:14.970 --> 00:27:22.710
William Cheng: And this physical memory location has to virtual address what is 01234567 what's going to be the corresponding Colonel virtual address. Well, we don't know yet.

195
00:27:23.580 --> 00:27:32.460
William Cheng: That. So we need to find the corresponding Colonel virtual address and what we try to write to this buffer, we better use the colonel virtual address because if you use the user space virtual address

196
00:27:33.090 --> 00:27:39.750
William Cheng: You might end up getting a user space, you might end up getting a kernel pays for all right now case of carnival crash.

197
00:27:40.590 --> 00:27:46.800
William Cheng: Okay, so therefore we need to do inside of Colonel because that we need to cover the user space virtual address into a kernel space virtual address right

198
00:27:46.950 --> 00:27:55.260
William Cheng: So how do we do that, right, so first we take this virtual so we're we're cutting into two parts, right, the virtual page number over here is going to be zero, x 12345

199
00:27:55.710 --> 00:28:00.420
William Cheng: Right. So in this case, the offset is going to be here. I'll say is going to be equal to zero, x 678 right

200
00:28:00.900 --> 00:28:09.060
William Cheng: Okay, so what we need to do is that we need to convert the virtual page number to a physical page number. How do we, how do we compare a virtual base number to a physical page number right

201
00:28:09.330 --> 00:28:15.030
William Cheng: All we need to do. So we need to find the right page frame because the page SRAM will tell us what the physical page number as

202
00:28:15.450 --> 00:28:28.710
William Cheng: Well guys, how do you find the physical with the patient object. Right. So what you will do is it will take the virtual page number walk down that linear list of his region and find out which memory segment it belongs to an inside a memory side where we need to find the page frame object.

203
00:28:29.790 --> 00:28:36.030
William Cheng: OK. So again, the purpose of the linear list of as region is for you to find out which memory segment at a pace that a patient belong to

204
00:28:36.240 --> 00:28:41.250
William Cheng: And then you ask that object to say find me the pace frame and it will give you a page, right. And then once you

205
00:28:41.460 --> 00:28:50.520
William Cheng: Once you have a Patreon. You can you can go to the patient object. Find the physical page number left shifted by 12 minutes at the offset to her and that will give you that will that will give you

206
00:28:52.470 --> 00:28:53.940
William Cheng: Sorry, I made a mistake over here.

207
00:28:54.300 --> 00:29:06.660
William Cheng: Well, so, so you find the patient object inside the patient object. There's a physical page number. There's also a kernel space virtual address for you to access this page right so in that case the kernel space virtual address will be a page a line address

208
00:29:06.900 --> 00:29:15.900
William Cheng: All you have to do is to add the offset to her and that will give you the colonel virtual address you know that the kernel virtual address for, you know, for for this buffer.

209
00:29:17.790 --> 00:29:19.830
William Cheng: Alright, so again inside your kernel three

210
00:29:20.310 --> 00:29:30.030
William Cheng: When we start implementing a hollow program. This is one of the code that you have to write to convert a user space virtual address to a kernel space virtual address. So you can actually write data into discover

211
00:29:30.690 --> 00:29:36.810
William Cheng: That whereas when you get there. So again, if you're confused, you know, feel free to send me email. Yeah.

212
00:29:38.040 --> 00:29:39.060
William Cheng: Alright, so we're gonna move on.

213
00:29:39.270 --> 00:29:46.950
William Cheng: So what we have more than one gigabytes of memory, right, if we have one more than one gigabyte of memory as you're going to get a little more complicated. So in this case, our real

214
00:29:47.100 --> 00:29:52.320
William Cheng: physical memory over here is going to be more than one gigabyte right here's one gigabyte over here. So maybe we have 16 gigabyte over here.

215
00:29:53.220 --> 00:30:00.840
William Cheng: So, so yeah, we're still talking about Linux. So for Linux what he would do is that it will take the colonel address space over here to that divided to par.

216
00:30:01.110 --> 00:30:13.140
William Cheng: The first Seven, eighth of the colonel address space over here. He's going to be exactly the same as we had before. And we're going to map a one for one to the first seven eighth of a gigabyte into the physical memory.

217
00:30:13.950 --> 00:30:22.380
William Cheng: Okay, so seven therefore Dubai, as it turns out to be 196 megabytes. So again, the first 700 896 megabytes is our current virtual address over here.

218
00:30:22.500 --> 00:30:29.880
William Cheng: Is going to map, map one for one, the physical address. So in this case, the physical address that we're going to be using is going to be the lowest, lowest portion of physical address space.

219
00:30:30.330 --> 00:30:38.100
William Cheng: There and the rest are one eighth of the, the, the game with you by over here and that's equal to 128 megabyte

220
00:30:38.940 --> 00:30:49.110
William Cheng: That so that part of the kernel of virtual address, which is that the end of the address space over here. They can be selectively map to any part of the physical memory using something called the K map.

221
00:30:50.400 --> 00:30:58.710
William Cheng: OK. So again, you know, in this case, the first seven eighth of the colonel address space is map in a fixed way to map the physical memory and that point. Whenever change then

222
00:30:58.920 --> 00:31:06.930
William Cheng: The last one eighth of the colonel virtual address over here can be controlled to map to a different portion of the address space. Okay, so in this example over here.

223
00:31:07.290 --> 00:31:13.530
William Cheng: Let me clear this up already. The last one eighth of the Colonel, you know, the colonel virtual doesn't we are is mapped to this portion

224
00:31:13.920 --> 00:31:20.910
William Cheng: Of the user space virtual dresser. So it says, That's right. It's not 120 828 megabytes or physical memory.

225
00:31:21.300 --> 00:31:33.270
William Cheng: Okay, if we want to access other part of physical memory and what, what can we do right, we cannot change the first seven, eight of the colonel virtual desk, but we can actually change the last part. So we can actually remap this to point to a different portion over here.

226
00:31:34.410 --> 00:31:40.800
William Cheng: Okay. So inside a kernel. If we need to access other part of the the physical memory that's other than the first 836 megabytes.

227
00:31:41.100 --> 00:31:52.170
William Cheng: Or we need to use to change the key map and devil can't use the top one eighth of the colonel virtual dress to access any portion of the physical memory, right. So this guy over here, we can remap it over here so we can actually change it to a different place.

228
00:31:53.880 --> 00:32:04.500
William Cheng: Okay, so, so this is all Linux does it right so starting from 196 megabyte if you want to access any part. Beyond that, you have to use the came out and you have to remap the kernel space virtual address

229
00:32:05.160 --> 00:32:12.060
William Cheng: That so Linux actually do is that if you buy physical memory over here into three different zones. So this is physical memory and not virtual

230
00:32:12.360 --> 00:32:23.040
William Cheng: The virtual memory that so the physical memory of there's going to be dividing two three zone over here. The first one over here is no as a DMA zone and that's for location sun for location zero

231
00:32:23.310 --> 00:32:36.300
William Cheng: All the way to 000 FF F, F, F, F, right. So this guy is there only 24 bit of the address are being used. So this is the VM is also where you need to transfer data from this into memory. You have to copy them into the DMA zone.

232
00:32:37.020 --> 00:32:46.440
William Cheng: So the DMA zone is pretty small. It was 24 bit address address is the reason for that is that because this is, you know, again, Linux home from Windows and Windows come from Doss

233
00:32:46.740 --> 00:33:00.840
William Cheng: In the good old days they are DMA devices that cannot use 3232 bit address on the bus. Okay. They can only use 24 hours on the bus, so therefore they can only transfer data into the first, you know, a 24 bit over here. It's a physical memory.

234
00:33:01.440 --> 00:33:06.510
William Cheng: Okay, so that's why are we here we dedicate those physical memory to build it and I saw that

235
00:33:07.680 --> 00:33:20.700
William Cheng: And then are the normal zone over here has come from the end. Starting at the end of the DMA saw all the way to 196 megabytes over here that these are called the normal zone because that's part that part of it is always dedicated to the Colonel.

236
00:33:21.240 --> 00:33:28.560
William Cheng: OK, so the normal side is not dedicated to the to the cut over the years as the always data structure must reside in this range.

237
00:33:29.400 --> 00:33:39.630
William Cheng: Okay, so if you have your pace table. If you have your kernel data structure, you know, your slot alligator, all that kind of stuff. They all needs to be sitting at a normal is all minus, you know, the DMA so

238
00:33:40.080 --> 00:33:42.750
William Cheng: Okay, so all the kind of data structure in a must read over here.

239
00:33:42.930 --> 00:33:53.280
William Cheng: So, so, you know, it's kind of funny for Linux over here. If you have only one gigabytes of memory, what the entire Colonel can actually fill up the one gigabytes of memory, we have more than one gigabytes of memory, the corrosion becomes smaller.

240
00:33:53.940 --> 00:33:56.430
William Cheng: Okay, the courage to become 196 megabytes.

241
00:33:57.240 --> 00:34:06.120
William Cheng: Or if you have more physical memory. OK. So the rest of the memory over here starting from 896 megabytes, all the way to the end of the physical memory. This is known as the high memes on

242
00:34:06.510 --> 00:34:09.660
William Cheng: So in this case is strictly you use for user pages.

243
00:34:10.620 --> 00:34:19.770
William Cheng: Okay, so this part of memory over here is completed by user space. Well, when so users. But what I'm going to be, it can be a use memory over here for the Colonel. The Colonel is not allowed to use data structure in the high Menzel

244
00:34:20.640 --> 00:34:25.590
William Cheng: Obey the kernel has have data structure inside normal zone and of course the tunnel can also use the DNA so

245
00:34:26.820 --> 00:34:29.430
William Cheng: Okay, so that's just how Linux is design. Yeah.

246
00:34:31.980 --> 00:34:39.780
William Cheng: Alright, so I guess there's X over here. So, so the the way the colonel manage the pace room over here is that, you know, again,

247
00:34:40.230 --> 00:34:49.290
William Cheng: For for the free pace. Read the free pace friends are used are managed by the buddy system. Right. So here are the three pages over here. So what they actually do is that they basically use three different

248
00:34:49.830 --> 00:34:58.140
William Cheng: Three different buddy system to manage each of the zones. Okay, so there's going to be a one buddy system for a DMS on one buddy system for an officer and one buddy system for high Mensa

249
00:34:58.440 --> 00:35:03.330
William Cheng: OK. So again, this is just how Linux designer system and other opportunities that may decide to do it to a different way.

250
00:35:03.570 --> 00:35:08.340
William Cheng: Now. So initially, there's going to be three different buddy system, each one of them manage your pre page rank.

251
00:35:08.550 --> 00:35:14.550
William Cheng: Well you allocate a free page or more, we here, they become an active page friend right and then we're going to, sort of, you know, we're going to use the

252
00:35:14.940 --> 00:35:25.200
William Cheng: Algorithm used to clock out for them to decide which pages frame is usually use them which Patreon is not recently use that. So again, the active pages over here are the ones that are picked up by the

253
00:35:25.800 --> 00:35:33.390
William Cheng: By the clock out where them as reason the US, so therefore they stay inside the active so you can sort of call this list of active page list.

254
00:35:34.080 --> 00:35:37.260
William Cheng: Okay, so this consists of all the pace right there are recently use

255
00:35:37.590 --> 00:35:41.520
William Cheng: When a patient of is not recently used you need to check whether the patient is dirty or not.

256
00:35:41.700 --> 00:35:49.260
William Cheng: If the patient is not dirty, which means that the patient has clear why not case, you can actually take out one of the patient. I'm over here and that send it to send it back to the best buddy system.

257
00:35:49.860 --> 00:35:54.450
William Cheng: Whereas, okay, the job of the page argument is to scan the occupation is over here.

258
00:35:54.660 --> 00:36:00.990
William Cheng: Find out which one is most recently, you know, it's not. If the patient is most recently used and stay inside the occupation as

259
00:36:01.170 --> 00:36:12.060
William Cheng: If they're picked out to the least recently us, what do we do that, you need to check whether the patient is dirty or not. If it's dirty. If it's not dirty. That means that exactly the same as the desk. So in that case, you will return them to the buddy system.

260
00:36:12.960 --> 00:36:18.690
William Cheng: That if it turns out the patient is dirty. Why, in that case what you need to do is that you need to remove them from the occupation us

261
00:36:18.870 --> 00:36:27.900
William Cheng: And move them into another list called the inactive page lists and the lack of faceless over here is going to get written out to the desk one page frame at a time and the pay that this is really slow.

262
00:36:28.110 --> 00:36:38.730
William Cheng: If you only have one day you can only run one page for a marathon. Right. So even though over here. There are three inactive patient as one from each zone, what in the end the disk and only right you know right these patients with this one at a time.

263
00:36:40.080 --> 00:36:50.460
William Cheng: Okay, so, so, so it is possible that over here that when your pace rainbow be here is being written out to the desk. What if your program tried to write to try to read this page right I'll try to write data to this patient.

264
00:36:51.420 --> 00:36:58.470
William Cheng: Was in that case going to be a little tricky, right. So if the patient I'm over here, turns out to be actively written out to the this, why don't you shouldn't really right to this page right

265
00:36:59.610 --> 00:37:06.150
William Cheng: Okay. So, therefore, what we need to do is that when we move a page for him over here from the actor page live today. Knock inaccurate tasteless over here.

266
00:37:06.360 --> 00:37:20.640
William Cheng: We're going to mark this page as busy to me that this page right now is busy to be written out to the this. So, therefore, if you have any user processes, try to use this page rank. They all have to block. They all have to wait until this patient because I'm busy, then it can be used.

267
00:37:21.870 --> 00:37:26.580
William Cheng: Okay. So, therefore, you know, this didn't know that the operations of here is going to be a little more complicated.

268
00:37:27.330 --> 00:37:28.650
William Cheng: That so that you're going to use the

269
00:37:29.100 --> 00:37:33.120
William Cheng: Algorithm to pick out the, the one that has r equals zero.

270
00:37:33.360 --> 00:37:45.510
William Cheng: And then also the dirty been over here is going to be able to what was the. In that case, that means that I need to write it out to the desk so therefore I will move this patients over here into the inactive patient is over here. And also, I need to set them too busy equal to one.

271
00:37:46.890 --> 00:37:57.090
William Cheng: Okay, so, so, so in this case, when a process or try to access, you know, this particular page right if the patient is is is busy that in that case that process, you have to go to sleep.

272
00:37:57.570 --> 00:38:02.700
William Cheng: Okay, eventually you know when this patient has been written out to the desk and now this patient.

273
00:38:03.180 --> 00:38:13.620
William Cheng: The data to be exactly the same as what's on the desk over here. So now, dirty bit over here, it's gonna be set to zero because no longer dirty, but in this case, do I return it back to the free page frame or not, then

274
00:38:14.130 --> 00:38:24.900
William Cheng: In that case, you know, it really depends. I actually need to see if there's any other processes that are waiting for this page rank if they're processing that are waiting for this page rank one. This guy this page rank become recently used

275
00:38:26.160 --> 00:38:33.270
William Cheng: Okay, if a process hasn't really usually yet. But if it's waiting for that one of them is is recently you. So in that case, I actually need to return it back to the actor page right

276
00:38:33.930 --> 00:38:41.790
William Cheng: So I need to return it back to the actor pays less if it turns out there's no Colonel thread that's waiting for a while. In that case what I should do is I should return into that buddy system.

277
00:38:43.290 --> 00:38:50.640
William Cheng: OK. So again, the life cycles over here for a patient over here is India. This is actually pretty complicated. Yeah.

278
00:38:51.450 --> 00:39:00.600
William Cheng: All right, here's sort of a summary. You know the full page over here, the free lists contain physical pages that have not been allocated they use the buddy system to maintain it.

279
00:39:00.810 --> 00:39:06.600
William Cheng: The active patients are the one that the content paste ran the picked up by the clock. Our them as recently use

280
00:39:06.780 --> 00:39:15.960
William Cheng: The inactive patron are the ones that are picked up by the clock algorithm as not recently us and also they are dirty or modified because they've they're not dirty. They can return back to the buddy system right away.

281
00:39:16.200 --> 00:39:25.860
William Cheng: Right. So if you're a dirty or modify. They are marked as busy and also they need to be a map from all the processes that are the sharing this page.

282
00:39:26.490 --> 00:39:37.380
William Cheng: Because I remember a page rank can be shared. Right. So if you move a patient over here, and instead it to be busy and if you don't have a map it from all the processing that we are one of the process. Want to use it and try to modify the data there.

283
00:39:37.860 --> 00:39:45.780
William Cheng: Well, so in that case the you know the data will be changing. If you're trying to buy the data onto this at the same time, while in that case, you know, something bad's going to happen right going to end up with memory corruption.

284
00:39:46.320 --> 00:39:53.610
William Cheng: Okay. So, therefore, what we do is that we need to a map this page frame from all the processes that are sharing the space. So how do you map it right

285
00:39:53.790 --> 00:40:02.130
William Cheng: So the way our map it is that you go to every page table that are using this particular page right and set the corresponding v equal to zero inside the page table entry.

286
00:40:02.730 --> 00:40:08.070
William Cheng: Okay, so this way when that, you know, will this work with that program tried to perform address translation.

287
00:40:08.250 --> 00:40:18.330
William Cheng: It will see be equal to zero. You're trying things out a kernel. And now when you are inside the Colonel, you will see that page. One is busy. So, therefore, you go to sleep and wait for it, wait for data transfer to get to get completed.

288
00:40:19.410 --> 00:40:26.730
William Cheng: That. So when you look on a page for me inside of the patient handler is the patient is Mike busy. You must wait until the day operations finish.

289
00:40:26.940 --> 00:40:36.750
William Cheng: Way for it to become I'm busy. And then you can use it right. So, therefore, this guy. You won't go sleep inside of Q when the data transfer as complete inside the IO completion interrupt you gotta unplug the Colonel.

290
00:40:37.470 --> 00:40:47.280
William Cheng: In this case, how many candles. Do you have to unblock, you have to unblock all the Colonel's threads that are waiting for this at this picture I'm over here, you need to wake up all the slides are waiting for this page, trying to become a busy.

291
00:40:47.700 --> 00:40:56.490
William Cheng: Guy. So this way you add them all back to the run queue. And then, you know, when they wake up. So in that case, you know, this page, but I'm over here will no longer be busy.

292
00:40:57.240 --> 00:41:06.510
William Cheng: OK. So again, it is possible, somehow the page demon getting between to be busy again, as I said, every time you wake up. You want to say, is this patient busy. If it's busy again. I need to go to sleep again.

293
00:41:07.350 --> 00:41:13.590
William Cheng: Okay, so yeah you know when you started your kernel three. There's a function called P frame get

294
00:41:14.850 --> 00:41:28.860
William Cheng: P frame underscore, get over here now. So again, instead of Venus kernel source. They all tried to name the function very reasonably consistently. The function of P for and get is very similar to the cinema as the function be get right

295
00:41:30.330 --> 00:41:35.790
William Cheng: Gradually, do you remember what what what what is the function v. Get right the V. Get over here is that, you know,

296
00:41:38.430 --> 00:41:44.820
William Cheng: Is that given the item number you try to get the page rank for that particular I know if the I know already, so sorry.

297
00:41:45.750 --> 00:41:50.970
William Cheng: The vino already exists that you find a vino increment the reference style, otherwise you're going to create the vino

298
00:41:51.630 --> 00:41:57.660
William Cheng: Okay, so over here P for and get it's the same thing, right. So we're here to try to look for a particular page rank if it's there already.

299
00:41:58.020 --> 00:42:02.370
William Cheng: I don't remember what the P friend object has a reference call maybe it has a reference count.

300
00:42:02.790 --> 00:42:10.080
William Cheng: I don't remember that. So so so necklace what he would do is that is that we were allowed to be shared. If it turns out that the you know the patient.

301
00:42:10.350 --> 00:42:19.740
William Cheng: The patient object doesn't exist. Well, in that case, it's the job of the pre frame get function to create the page frame, you know, for, for, for the page that you're looking for.

302
00:42:21.000 --> 00:42:29.310
William Cheng: Now, guys. So again, this is how do you create a page for him over here. Right, so, so, so in this case if the patient doesn't exist. And this page ran the data need to confirm that this one. This guy is going to

303
00:42:29.580 --> 00:42:32.880
William Cheng: Create the patient magic and he also need to transfer data from a distant memory.

304
00:42:33.300 --> 00:42:43.890
William Cheng: That's okay. All these things are pretty complicated. So, so again inside your carnal three. The first three function, you have to implement is P for and get p for in pen and paper and I'm paying them talking about pin and I'm paying a little later.

305
00:42:44.340 --> 00:42:52.710
William Cheng: But this will be the first one so that you have the implant. Again, you need to think about this particular case, you know, because he's a handle the logic correctly. Now,

306
00:42:54.390 --> 00:43:00.900
William Cheng: Oh, I mean, some people doesn't really do this correctly in the beginning, it's okay because your chances are your kernel code, code, code, code.

307
00:43:01.230 --> 00:43:10.290
William Cheng: Code is going to work when your current always not too complicated, but later on when things are complicated that you have to, you have to make sure that your code is done correctly. Okay, so just because you're co run

308
00:43:11.190 --> 00:43:17.520
William Cheng: It doesn't mean that you that you implement is correct, maybe just because you're lucky. Okay, so that can certainly happening. So your assignment. Right.

309
00:43:19.320 --> 00:43:29.820
William Cheng: Okay so page frame. It's pretty complicated over here. There. Alright, so, in which case the other way, they implement the user address space over here, we talked about, you know, the outer space data structure.

310
00:43:30.450 --> 00:43:36.330
William Cheng: That that data structure basically look like Linux address space implementation, right, we have a tech segment data side we have BSS plus

311
00:43:36.990 --> 00:43:47.100
William Cheng: Sorry overhead BSS data SPSS like men. Right. And then we have a dynamic segment over here. And then there's a big gap. And then we have a stack at the bottom. Over here, right. So get the data structure over here.

312
00:43:47.670 --> 00:43:52.470
William Cheng: Inside we need, they just give it a different name, but the the sort of the, the, the, the structure is the same.

313
00:43:52.800 --> 00:44:01.710
William Cheng: I guess the instead of calling them process control blog, they call it a tests task struck instead of calling a virtual memory, the map. They call it the memory map struck

314
00:44:02.160 --> 00:44:03.510
William Cheng: Okay, instead of calling them.

315
00:44:04.140 --> 00:44:16.380
William Cheng: Sort of as region and address space region they call the VM area. So again, area memory segment you know region. They all mean the same thing. It's contiguous virtual address location that have similar data that's inside of it.

316
00:44:16.680 --> 00:44:26.850
William Cheng: OK. So again, this is the text segment attacks VM area. There's a data plus BSS VM area. Here's the heat VM area. And then over here, here is the

317
00:44:27.690 --> 00:44:35.490
William Cheng: The stack VM area. So again, you know, things that are that are related to the pharmacist, then

318
00:44:35.910 --> 00:44:42.990
William Cheng: They there's a point to point. The default logic over here, things are not have the backing store instead of passes them. In this case, they will plan to anonymous object.

319
00:44:43.560 --> 00:44:48.210
William Cheng: OK. So again, this is the data structure that's used by Linux and also inside a VM area over here.

320
00:44:48.390 --> 00:44:59.670
William Cheng: We have to start address with the length of the memory segment, we have what kind of protection. It has also whether the memory segments shared private all the stuff we talked about before, they'll actually use in Linux. Okay.

321
00:45:01.320 --> 00:45:05.430
William Cheng: All right, you map a failing to address space you're going to end up with a new memory segment.

322
00:45:05.730 --> 00:45:12.330
William Cheng: So over here, again, you create a sort of a memory map out inside your memory segment. So in this case over here, you're gonna get a new memories, like, man.

323
00:45:12.660 --> 00:45:16.830
William Cheng: If you create a lot of our memory my file you create also where you create a

324
00:45:17.310 --> 00:45:22.590
William Cheng: New thread a user's Facebook when you can end up with a news or stack segments or this guy again, you

325
00:45:22.800 --> 00:45:24.420
William Cheng: Get the link has become longer and longer.

326
00:45:24.600 --> 00:45:37.770
William Cheng: So, so in this example over here we have 117 map out. We have tons of stuff over here. So, in that case that linear list of VM area is going to become really, really long right so now I'm going to sort of call them VM area. He has region though all interchangeable. Okay.

327
00:45:37.980 --> 00:45:45.150
William Cheng: So that link is going to be really, really low. And every time when you when you get a paste all you need to walk down that linear list over here and that will take you a long time.

328
00:45:45.600 --> 00:45:52.380
William Cheng: Okay, so he Linux what they actually did over here is to implement that as a tree data structure. So this way again where you are, we cover the patient call

329
00:45:52.860 --> 00:45:58.800
William Cheng: Us at a pace all the first thing you need to do is that you need to look for the memory segment that the container virtual dress.

330
00:45:59.160 --> 00:46:09.630
William Cheng: Okay, so when you have a tree data search over here, you can actually find the memory segment over here in login time what. And there's a number of memory segment. So again, the performance going to be much better than order n.

331
00:46:10.020 --> 00:46:18.240
William Cheng: You know when you have a linear list. That's okay. This is what Linux I implement it again. The look up there and check it doesn't really matter whether you allow us a list. Use a hash table use

332
00:46:19.080 --> 00:46:26.730
William Cheng: You know, use a binary tree or some other kind of tree over here. It doesn't really matter because that's a data structure that all you have to

333
00:46:27.150 --> 00:46:40.620
William Cheng: Look out, given a virtual address find which memory segment that owns that virtual address. OK. So again, the bread, you know, the memory segment. They're all destroying that all app, right. So, therefore, again, you can use data structure like this to try to find that memory segment. Okay.

334
00:46:42.810 --> 00:46:53.730
William Cheng: Oh I on the Linux I'll paste memory scheme over here. So here's a sort of quick summary. Are they use it without getting into too much detail they use a two hander clock algorithm. It's going to apply to the zones in sequence.

335
00:46:54.150 --> 00:47:02.070
William Cheng: Guy so so they will go, they will go to DMS on first and then they will go to the normal and they've got the high Menzel using a to hand o'clock hour then

336
00:47:02.460 --> 00:47:07.860
William Cheng: They also they use global allocation and they don't use local allocation. Again, this is in contrast with Windows

337
00:47:08.250 --> 00:47:16.590
William Cheng: Which does things a little differently. Right. So again, the picture was sort of talking about it already, so I'm not going to go over again right you know you you know the the active page frame over here.

338
00:47:17.130 --> 00:47:26.940
William Cheng: You know it's going to be the one with the reason the US and the inactive page ram over here is going to be not recently used and and they're dirty. So therefore, there you know you said he's a bit and all this kind of stuff. Yeah.

339
00:47:28.050 --> 00:47:34.470
William Cheng: So again, the page. I'll demon over here is to handle it. Right. So the first time when he wakes up, it's going to go to these, you know, these zones in sequence.

340
00:47:34.650 --> 00:47:45.780
William Cheng: And then what it will do is it will do all the stuff I mentioned before, and then we'll go to sleep and then when the second one, you know, the second one we wake up over here. Again, it's gonna go to all these only sequence over here. And then, and then what it will do is it will

341
00:47:47.490 --> 00:47:52.590
William Cheng: Do for for the patient in that are not recently use their dirty and then they will move that into the next page lists that

342
00:47:55.410 --> 00:48:02.400
William Cheng: Are here are some important or Linux data structure, sort of a summary and he sort of example who are using all these data structure.

343
00:48:02.610 --> 00:48:11.820
William Cheng: Okay, so for every process. The process control blog and we saw the linear, they call it a test struggle over here. Again, the name doesn't really matter what's important is, you know, sort of the, the concept

344
00:48:12.360 --> 00:48:15.300
William Cheng: They have a virtual memory map. And they also have a hardware paste table.

345
00:48:15.510 --> 00:48:24.030
William Cheng: The virtual memory map is used to represent the user address space and doesn't contain the colonel address mates, right, because the colonel doesn't really need to address space representation. Okay.

346
00:48:24.630 --> 00:48:33.510
William Cheng: They will map virtual memory segments or as a color, the color BM area, they keep track of page for him and the backing store right again the page where we're at the pace ram that

347
00:48:33.990 --> 00:48:43.290
William Cheng: The, the other the recap inside of the memory segment. Right. So again, once you find the VM era data structure from the VMware data showed you, you can actually try to find a PageRank.

348
00:48:44.070 --> 00:48:50.760
William Cheng: Okay. As it turns out, there's a little complicated, and the next part of this this lecture. What sort of talk about exactly how that is done then

349
00:48:51.570 --> 00:48:59.010
William Cheng: All right. And also, you know, they have a backing store why we can use the file object or the anonymous object to keep track of data in case they need to go to the

350
00:48:59.370 --> 00:49:07.890
William Cheng: Go to the back end. Store the backing store can be inside the file system hierarchy or can be inside the heap region that, sorry, sorry, or inside. Is that a soft space.

351
00:49:08.100 --> 00:49:13.500
William Cheng: Right, if it seems out of phosphate. They are managed by the anonymous object, otherwise it are managed by the file object that

352
00:49:13.980 --> 00:49:20.310
William Cheng: There's also the hardware paste a ball over here. Every page table. There's a page table entry, you need to set them and then this way we can control.

353
00:49:20.730 --> 00:49:24.600
William Cheng: You know, which access with caution because baseball and which is why they're

354
00:49:25.290 --> 00:49:34.620
William Cheng: Going every, you know, to, to, to go across all the processes. Right. You know, these are process data structure. And then there's global data structure or their shared by all the processes.

355
00:49:34.980 --> 00:49:43.770
William Cheng: They are the free list the occupation is saying occupations that we just saw that we just saw. Yeah. And here are a couple of quick example how this data structure to be used.

356
00:49:44.160 --> 00:49:53.130
William Cheng: For example, what happened with the pace Welker again. There are many reasonable pace fall can be be equal to zero. You can be trying to write to a read only page or could be you're doing

357
00:49:53.340 --> 00:50:00.270
William Cheng: You're doing this on purpose because you doing copy on write it says, Okay, we're going to start again summarize this using an example where the equal to zero.

358
00:50:00.780 --> 00:50:05.610
William Cheng: So this guy pays for all come from the hardware is equal to zero. So, in this case to come from the menu.

359
00:50:06.000 --> 00:50:11.910
William Cheng: So we're going to be inside the Colonel. The Colonel is gonna you know it's going to ask the buddy system for new page for it.

360
00:50:12.480 --> 00:50:18.960
William Cheng: So if the buddy system say they're all out. Chances are, if you have a good page ID man, but he says insurance, you should never run a PageRank.

361
00:50:19.380 --> 00:50:29.880
William Cheng: Well, so, and this guy. Want to get a Patreon Patreon from the buddy system. And then what we need to do is that we need to mark the page from as busy and then we're going to copy the data from the desk into memory.

362
00:50:30.630 --> 00:50:37.950
William Cheng: There. And then what would this operation is done. We're going to adjust the horror paste able to adjust all the colonel data structure and then it's going to be very

363
00:50:40.260 --> 00:50:44.100
William Cheng: In this case, we're going to return back to the user space forward. Yeah.

364
00:50:44.790 --> 00:50:51.000
William Cheng: The basic idea is pretty straightforward. But again, remember that a page from over here can be shared by multiple processes.

365
00:50:51.180 --> 00:50:58.830
William Cheng: So in this case, just because you've got to be equal to zero inside your user process when he comes out of Colonel maybe the patient already exists inside, Colonel.

366
00:50:59.340 --> 00:51:08.400
William Cheng: Okay, so maybe that patient is used by other processes and policies. So in this case you should try to look for it first easy because if we can find it. All you have to do is to share it.

367
00:51:08.820 --> 00:51:14.550
William Cheng: Right. So again, for doing copy on write, you know, in this case, we can actually share it all. We want until you tried to write to it for the first time.

368
00:51:14.820 --> 00:51:22.980
William Cheng: So you reality, it can be more complicated because the patient can be sure. Okay, so you don't necessarily have to ask the buddy system. Every time will you get a page Hall.

369
00:51:23.520 --> 00:51:33.990
William Cheng: Okay, if you can find that you use it if you can't find it. In that case, when you copy for engage you need to create a new page. Read Only at that time you ask the buddy system for new patron right otherwise you will share that

370
00:51:35.460 --> 00:51:39.810
William Cheng: By the seven users or here's what happened when the pace or demon wants a free out of

371
00:51:40.470 --> 00:51:48.060
William Cheng: Free up a modified the 30 page over here, right. So again, the page or demon. Oh, we are pick out a page friend to be

372
00:51:48.390 --> 00:51:56.520
William Cheng: To be now recently us, as it turns out, the page frame is is dirty, because if the patient is clean, they would just return to the buddy system. Right. So in that case will be very straightforward.

373
00:51:56.820 --> 00:52:10.590
William Cheng: Okay, otherwise you'd be reaching out to the desk. Right. So in this case, what we have to do, right, we need to find out which processes and address spaces over here, we use the plural form over here intentionally because the page rank can be shared by many processes in many

374
00:52:11.310 --> 00:52:13.800
William Cheng: In many addresses that. And then we need to

375
00:52:14.040 --> 00:52:24.090
William Cheng: Map. The pages from the corresponding page table right for all these processes are sharing that page drag since we need to cut we needed to transfer that data to this over here, we need to make a busy. So we need to

376
00:52:24.330 --> 00:52:31.770
William Cheng: Map it from all these address basis over here, right. So in this case, we need to share it says that all them equal to zero for all those paper entry over here.

377
00:52:32.370 --> 00:52:42.840
William Cheng: I'm going to recommend you to read a piece of code inside we next race, it means that we need. There is a function call P frame remove from page tape of a front page tables.

378
00:52:43.410 --> 00:52:50.100
William Cheng: That. So there's one function that will remove a patient that will a map a paste ran from all the address based spaces are using it.

379
00:52:50.550 --> 00:53:04.650
William Cheng: Okay, so if you read this code. This code is I think it's less than 10 lines long. Okay, it's a very powerful functionally what a map, you know, this page table entries over here from all the address spaces that are using this particular page using this particular page right

380
00:53:05.850 --> 00:53:11.820
William Cheng: Guys. Okay. So, so what this you know this function over here is inside the code that you are not allowed to modify

381
00:53:12.300 --> 00:53:17.790
William Cheng: Okay. So, therefore, what what is telling you is that your call. When you try to write your kernel three call it must work with assumption.

382
00:53:18.270 --> 00:53:25.950
William Cheng: Well guys, when you build the I just may say to structure it must work with this function, because otherwise you know when you try to remove a pace rain from outer space. That won't work.

383
00:53:27.240 --> 00:53:37.140
William Cheng: Okay, so your requirement is to build the address space so that all the existing Colonel all the existing pristine kernel source code, they will all work. So one way to understand it is by looking at this function over here.

384
00:53:38.220 --> 00:53:48.720
William Cheng: Okay. So, this function is you know 1010 lines of code, right. So, so by trying to understand it, you actually understand some of the fields inside the patient map jag and also means that other object inside the kernel source code.

385
00:53:49.440 --> 00:53:52.410
William Cheng: That's okay. I highly recommend you to try to understand the code there.

386
00:53:53.430 --> 00:54:06.090
William Cheng: Alright, so in that case. Why should a map them all down from, you know, for, for all the address spaces that are using it. Now it's safe to write it to the desk or so this case, you're going to find the corresponding backing store. Right. But again, using your

387
00:54:06.450 --> 00:54:12.120
William Cheng: You know your linear list of VM area over here. So in that case, you will be able to find whether you should use

388
00:54:13.080 --> 00:54:17.100
William Cheng: Us a file object to write for back install user, not on the sockets right into the back end so

389
00:54:17.520 --> 00:54:25.410
William Cheng: That was all. And then you need to set the patient to be busy. So this way when any of the user space. We're gonna try to use it, they'll have to go to sleep.

390
00:54:25.860 --> 00:54:30.720
William Cheng: That, in the end, you know, when the data is transferred from the desk when it

391
00:54:31.230 --> 00:54:35.700
William Cheng: Will not appreciate is that what you need to do is that you need to check if any processes waiting for or not.

392
00:54:35.940 --> 00:54:48.360
William Cheng: If any process waiting for a while, then this patient become recently used. So you need to wake up with all those processes. If know processing are waiting for them. He said, nobody needs that anymore. So in this case you can free the pace frame and return it back to the buddy system.

393
00:54:49.440 --> 00:54:54.690
William Cheng: OK. So again, all these complication over here has come from the fact that the patient can be shared by multiple processes.

394
00:54:59.100 --> 00:55:06.510
William Cheng: Alright, so this is actually a good point to break. We just finished up on top of our Linux. And next time we'll sort of briefly take a look at windows. Okay.