WEBVTT

1
00:00:04.920 --> 00:00:15.750
William Cheng: Hey, this is the second part of lectures 17 so we just finished talking about the log structure passes them. So now we're going to briefly take a look at windows for Windows, the actual POS system.

2
00:00:16.170 --> 00:00:27.870
William Cheng: I guess in the good old days, it's fast 16 and fast 32 fast dance will file location table. So the way the windows allocate this block, they actually use the first fit alligator.

3
00:00:28.470 --> 00:00:38.400
William Cheng: So it's very inefficient and also have your external fragmentation sort of how all kinds of trouble. Okay. Also, their data structure for them to, to, to keep track of.

4
00:00:39.480 --> 00:00:43.410
William Cheng: The, the files. Hold on a second.

5
00:00:45.630 --> 00:00:52.770
William Cheng: Okay. They call it the, the, the extent data structure. So, remember this is the boss is them. There's something called this map that this map.

6
00:00:53.190 --> 00:01:02.580
William Cheng: In the that's the one has a direct point or the grad point or the doubling or tripling direct. Those are the pointers that point to the actual this blog that store the data for file. Okay. In Windows

7
00:01:03.030 --> 00:01:15.000
William Cheng: The data structure. They call it the extent data structure. So an extent it's a list of runs the runs is a consecutive this blocks right so for example over here. Here is a running list. It's a list of runs

8
00:01:15.570 --> 00:01:21.300
William Cheng: So in this case, what is trying to say that you know the the first run. It's a length of three blocks.

9
00:01:22.050 --> 00:01:39.270
William Cheng: Okay. And the first block starts at block number 117 to eight. Okay, so that means that in order for you to get the first three blocks of this particular file, you go to block number 11728117 to nine and 11730 right so that's block number 01 and two. Okay.

10
00:01:39.690 --> 00:01:56.550
William Cheng: What about the next four blocks the next four block starts at 10624 so block number three is at 10624 and then followed by 11062510626 and then 106 to seven. So that will be 3456

11
00:01:57.030 --> 00:01:59.400
William Cheng: So, so this is one run this is one round. So again, the

12
00:01:59.850 --> 00:02:10.470
William Cheng: Run is a consecutive this block numbers. Okay, so, so you can actually, you know, so, so this one I will give you the first three blog and this will give you an ex football and then this won't give you the next few blogs and the next next week.

13
00:02:11.190 --> 00:02:17.280
William Cheng: Well guys will happen is that when you try to create a file, you create a file, a little bit at a time. So that's how these fall blocks are allocated.

14
00:02:17.790 --> 00:02:26.010
William Cheng: And they keep everything will be getting started wrong this. So the problem with the run this is that the runners will get very, very long. So in this case, if you want to, you know,

15
00:02:27.000 --> 00:02:30.450
William Cheng: In this case, if you want to look for some some some some some block number

16
00:02:30.810 --> 00:02:40.530
William Cheng: Inside your, your particular file, you might have to do a very, very long seek by by retrieving this entire run this. Okay, so, so in this case the wrongness is going to be stored like a file.

17
00:02:40.740 --> 00:02:47.940
William Cheng: And this file can get really, really big. And in the end, when you finish reading the run this all you will all you will get as a bug number and then finally use that block number

18
00:02:48.330 --> 00:02:55.440
William Cheng: To sort of figure out where to go to this to get that piece of data that. So this is the this is the extent data structure that's using

19
00:02:57.450 --> 00:03:06.990
William Cheng: So again, you know, if the length of this list is is then whenever you try to look for data blog inside this file, it's going to cost you order at night. So it's, again, this is really, really slow.

20
00:03:07.530 --> 00:03:18.270
William Cheng: So here's a reminder of what this is the purposes of this map look like right started out with the, you know, inside the I know we have this map, and that this map again. The first 10 is direct pointer, followed by indirect double indirect

21
00:03:18.540 --> 00:03:22.950
William Cheng: And direct and then the blue parts over here is actually the data. The data for the file.

22
00:03:23.250 --> 00:03:30.420
William Cheng: Okay, in contrast to Windows. These are the blue box over here. These are the data, you know that these are, these are the date of the blocks.

23
00:03:30.600 --> 00:03:35.700
William Cheng: That sort of data for your for the file. But in order for you to get to this data, you have to go to the run this first

24
00:03:36.060 --> 00:03:42.270
William Cheng: OK, so the problems that the roundness can be very, very long in order for you to look for a particular blog, you might cost you order in

25
00:03:43.080 --> 00:03:51.840
William Cheng: The order number this greets you know before you can actually figure out where the data is. Okay, so that's one of the major problem with the performance in fact 16 and 32

26
00:03:52.290 --> 00:03:57.030
William Cheng: For those of you who use Windows or you will see that the Windows file system is actually really, really slow. Okay.

27
00:03:57.870 --> 00:04:03.960
William Cheng: All right, so there are two problem. You know, so two major problem with the fast 1632

28
00:04:04.260 --> 00:04:14.010
William Cheng: You know process that number one is that since they use the first good memory allocated algorithm to allocate this blog. They're going to have the same problem as the first rebel were then and that will be with the external fragmentation.

29
00:04:14.340 --> 00:04:21.660
William Cheng: OK. So again, those of us who use Windows. You will notice that once in a while, you have to run a program called the different mentor.

30
00:04:22.200 --> 00:04:27.930
William Cheng: Okay, so I can decide, you know, if you look at this right here is is block number 012 all the way to the end of the desk over here.

31
00:04:28.650 --> 00:04:34.440
William Cheng: That, you know, we use the first foot alligator on the, on the, this data, you're going to end up with

32
00:04:34.680 --> 00:04:40.890
William Cheng: You know holes all over the place over here where the you know the the the frivolous, but I have a whole and those holes are so small that they're not usable.

33
00:04:41.490 --> 00:04:49.410
William Cheng: OK. So the purpose of it. The frog winters, those, those of you who had to run it, you will see that what it will do is it will actually shuffle all that this data around so that all these

34
00:04:49.680 --> 00:04:57.240
William Cheng: You know, external fragmentation. They all go to the end of the desk and they will push all the data until we get to the beginning of a desk. Okay, so if you combine all the

35
00:04:58.080 --> 00:05:02.610
William Cheng: So, so if you combine all the other the block. There are too small to be used.

36
00:05:02.790 --> 00:05:12.270
William Cheng: And you push them all the way to the end of it is. So in the end, you can end up with one giant free blog at the end, while all the other you know what what what all the alligator blah, they'll go to the beginning of this

37
00:05:13.110 --> 00:05:21.270
William Cheng: Now, so will you. Rather, the front window, you're going to get this nasty message on the screen to say, be very careful. Don't turn off the machine you can imagine that if, if, in the middle we turn off the machine.

38
00:05:21.480 --> 00:05:27.870
William Cheng: Part of the discovery has been updated and then the other parts over here are actually in limbo or in this case the entire. This might not be usable.

39
00:05:28.470 --> 00:05:34.950
William Cheng: Okay, so every time when I wrote the frog mentor. I was very nervous because what if I lose power. What if I there's operating system bug or something like that.

40
00:05:35.280 --> 00:05:38.190
William Cheng: In that case, I'm going to end up losing my entire desk that

41
00:05:38.700 --> 00:05:45.180
William Cheng: So this is a really terrible thing to us. So, even today, you know, we were you running Windows, you still have that the fragment your desk once in a while.

42
00:05:45.540 --> 00:05:48.060
William Cheng: Because they are using the first word memory alligator now.

43
00:05:48.540 --> 00:05:56.160
William Cheng: The second problem is I will be here is that, you know, when you perform a random search right you perform a Sikh and then you try to do a read that will cost you order.

44
00:05:56.490 --> 00:06:04.260
William Cheng: The order n, because the wranglers can be very, very long. It can spend many, many this blog. So in order for you to find out where is Brock number 10,000

45
00:06:04.980 --> 00:06:06.420
William Cheng: Okay. So this example over here. I

46
00:06:07.260 --> 00:06:12.150
William Cheng: Will show you the beginning over here. So let's say that you try to go to block number 10,000 where's the block number 10,000

47
00:06:12.360 --> 00:06:21.900
William Cheng: You have to do a sequential search on the beginning over here by adding up all these length blog and eventually you can figure out, you know, well, well. What a pretty good blogs blog update is

48
00:06:22.200 --> 00:06:28.710
William Cheng: OK. So in this case, you have to go to this many, many times. And that's why slow down the performance. Okay, so for for random access on the file.

49
00:06:29.250 --> 00:06:34.110
William Cheng: The, the fast 16 at 532 they're actually terrible and we mentioned before for system five assist them.

50
00:06:34.290 --> 00:06:42.240
William Cheng: In the worst case, all you have to do is to do some point to math and figure out exactly where things are, and read these three data blocks over here and you can find out what you need to go.

51
00:06:42.810 --> 00:06:53.880
William Cheng: Okay, so therefore, for system for file system. This operation for a random search as well for a random seek is going to cost you order one well on the system of a while on Windows is going to cost you order at

52
00:06:54.600 --> 00:07:00.720
William Cheng: That so so in order for windows to solve the, you know, to, to solve the performance problem.

53
00:07:01.560 --> 00:07:10.110
William Cheng: So, so what is this what a solution right there. Again, this is a classical computer science problem, you'd be have a link lists of water and the solutions, the years to build a cheap a hash table.

54
00:07:10.560 --> 00:07:21.360
William Cheng: So well window was that they used a multi level three data structure in order to store a store the runners, then. So basically what they do is that instead of using linear runners. They use a two level roundness there.

55
00:07:22.170 --> 00:07:36.270
William Cheng: Whereas over here, there's usually two levels. I don't know if you can actually go to more than two level, but typically to is to should be enough. So the idea here is that use the to level one less. Okay. And the trick. Over here is that everyone list has to fit inside one this block.

56
00:07:37.680 --> 00:07:48.900
William Cheng: Okay, so this way in order for you to look for some data first. You go to the first level runners over here on the first level runners will point to a second level around this. So for example over here. The second level run this over here, it actually I

57
00:07:49.260 --> 00:07:55.260
William Cheng: Can actually store information about where the first 84 blocks of the folders so in the top level Rome is over user

58
00:07:55.470 --> 00:08:07.260
William Cheng: In order for you to find the first 84 blocks inside this file, go to this way around this. Right. And inside this run this again, this is one, this blah. So all you have to do to perform run one read of the desk and you can figure out exactly what it what data is

59
00:08:07.530 --> 00:08:14.700
William Cheng: There so so good when you start to perform a C corp, followed by Re. Will you do a Sikh and then what you do a read, you will read the card level runners.

60
00:08:14.910 --> 00:08:22.650
William Cheng: And have all these numbers over here you can sort of figure out where it was which second level relevance to go and then you read the second level around this and it will tell you where the blocks are

61
00:08:23.460 --> 00:08:30.600
William Cheng: Okay, so this way in order for you to get one data blocks all all you have to do is to go through to run this and you guaranteed to be able to find the data.

62
00:08:31.110 --> 00:08:37.230
William Cheng: OK, so again. Well, that's a guarantee is that you know is that because typical of the run. This is only two level. If you can get the three level, whether in that case.

63
00:08:37.830 --> 00:08:43.170
William Cheng: In that case is going to require you to do three this week. So I think three level run this is very, very rare.

64
00:08:43.680 --> 00:08:48.060
William Cheng: Okay. So typically, you can actually get to all the data using a tool that will run this there.

65
00:08:48.870 --> 00:08:51.720
William Cheng: So in this case, you know, so we're gonna end up with better performance, right.

66
00:08:52.050 --> 00:09:00.570
William Cheng: That the performance over here is actually going to be better than the system Barbosa because this isn't about our system I we saw before. In the worst case, in order for you to say, okay, you know,

67
00:09:00.780 --> 00:09:10.380
William Cheng: Let's compare this picture over here. So, so, you know, over here, this is a run this then run. This is similar to the to the this map. There's over here we have two levels of this map over here.

68
00:09:10.920 --> 00:09:14.430
William Cheng: Those are the part of the the farthest or the metadata.

69
00:09:14.700 --> 00:09:21.570
William Cheng: And then the blue part is the actual data. So if we really try to get to the the blueprint of the data in the worst case for this is above us is that

70
00:09:21.750 --> 00:09:31.470
William Cheng: You have to read one that I did a blog and the second minute in a blog and the Thurman identify and then you can figure out whether whether the last day of buckets. Right. So in this case, the overheads going to be three data blocks that you have to read

71
00:09:31.980 --> 00:09:39.390
William Cheng: For the multi level run this for Windows. In this case, you will you will have to be to data loss and that will tell you where the final data is

72
00:09:39.990 --> 00:09:48.240
William Cheng: OK. So in this case, you know, instead of going to do this three times. You don't have to go to this two times. So in this case, your performance is going to be increased by 33%

73
00:09:48.960 --> 00:09:58.200
William Cheng: Okay so 3% is actually pretty good. So we're going to see a lot of the more modern operating system inside the fastest and they will actually use extent as their data structure. Yeah.

74
00:10:00.060 --> 00:10:10.380
William Cheng: Alright, so that's all I want to say about a performance. The next topic that wants to talk about is called cross resiliency. We want to make sure that when your file system crash.

75
00:10:11.130 --> 00:10:17.790
William Cheng: You know, we, we will be able to recover right we saw before, you know, using the, the larger file system.

76
00:10:18.210 --> 00:10:23.610
William Cheng: If you use the if he is a sort of a double buffering trick by using to checkpoint file.

77
00:10:24.030 --> 00:10:37.290
William Cheng: Whenever you get a crush all you're going to lose is the last, you know, several seconds of data and then you and then you'll be your first step is guaranteed to be in a consistent state then. So now we're going to see how to do this in general for any kind of a file system back

78
00:10:38.640 --> 00:10:39.750
William Cheng: Alright, so

79
00:10:40.080 --> 00:10:44.910
William Cheng: So then we're going to assume that we're going to have a we're going to assume that we're going to use a large buffer cache.

80
00:10:45.030 --> 00:10:50.940
William Cheng: And we're going to use the right back mechanism because the right Bob records that will make it look like your file system is very, very fast. Okay.

81
00:10:51.120 --> 00:10:56.670
William Cheng: If you don't use right back the performance going to be really bad. So therefore right back is sort of a requirement, these days.

82
00:10:57.060 --> 00:11:02.700
William Cheng: Okay, so, so, so we're going to use a buffer cache whether whether we perform the can read. Read a simple. We don't have to worry about re

83
00:11:02.880 --> 00:11:11.610
William Cheng: When you provide the right operation you're going to modify the data blocks inside the proper cash. So the pink broccoli that I showed over here, right. Those are the ones that are dirty.

84
00:11:12.000 --> 00:11:20.580
William Cheng: Because remember the definition of dirty rat the dirty blocks or the one that's different from what's on the desk. Right. So in this case, once they are not dirty, what you will do is that, you know,

85
00:11:22.200 --> 00:11:30.930
William Cheng: What would you do that you will release all these 30 blocks to a to a thread inside of kernel. And that's what it will do is that I will take all these 30 bucks and write it back to the desk.

86
00:11:31.530 --> 00:11:36.750
William Cheng: But so that the task on the inside corners know is that this update task or that this updates.

87
00:11:37.200 --> 00:11:42.330
William Cheng: So what it will do is that it will move for all these 30 blocks. And then what it will do is it I will read about back to this.

88
00:11:42.510 --> 00:11:51.270
William Cheng: Or the other way you can sort of think about is that as soon as you Mark. Mark abroad 30 you will actually you edit to the queue for this update has and that this update is we'll take all these

89
00:11:51.630 --> 00:11:56.310
William Cheng: You know, all these parts over here trying to sort of figure out what is the best way to write them out onto the desk.

90
00:11:57.210 --> 00:12:05.130
William Cheng: Right. Alright. So again, this only applies to the dirty blogs over here again that 30 blocks is going to be. It's going to be right right right right to the desk, are you

91
00:12:05.610 --> 00:12:16.530
William Cheng: Using the right mechanism will be writing, writing out to this at a later time. Right. In the meantime, you gotta return from right so at this point. If you have only written out two blocks of this and now you get an operating system crash.

92
00:12:16.740 --> 00:12:26.580
William Cheng: Or you lose power like that case your file system will become inconsistent. Okay, so therefore we need to have a way to make sure that the file system is not going to be, it's not going to an inconsistency. Right.

93
00:12:28.770 --> 00:12:36.540
William Cheng: Okay, so, so, so let's take an example to see how you know what kind of bad things can happen. There is a. Here's an example over here.

94
00:12:36.750 --> 00:12:43.260
William Cheng: This is the state that we're going to start with. Okay, we're gonna start with one of the display over here, this, this file has appointed that point or something.

95
00:12:43.470 --> 00:12:50.340
William Cheng: And this is going to be the final state of the file system. Right. Again, this is the actual process that we're going to write it down to the desk. So the final state over here is that

96
00:12:50.700 --> 00:12:57.030
William Cheng: Is that inside of us as though we're going to modify to this blocks. Why is this is the wisdom of this work over here. And the other one is new.

97
00:12:57.540 --> 00:13:07.080
William Cheng: Okay, we're going to modify this this part over here to point to a new this blog over here. The problem over here is that we want to write these two blocks out to the this using one atomic operation.

98
00:13:08.790 --> 00:13:16.320
William Cheng: Okay, we know about coming operational right the time operation. All you have to do is to lock things together and then you write the data out. And then you can actually unlock the data.

99
00:13:16.560 --> 00:13:25.530
William Cheng: Apparently, that doesn't work with a desk. Why doesn't it work with the desk right over here, because when I talk about system crashes. There's no way for you to implement a log and tell the system, not to crash.

100
00:13:26.280 --> 00:13:32.160
William Cheng: Okay, because if you have an operating system Barbie this system crash. There's no way for you to say, hey, wait until you know I finished writing out to the desk.

101
00:13:32.550 --> 00:13:41.310
William Cheng: So, therefore, you know, there's no way to perform. You know, you know, the to take you from one to four here step why his stuff over here. There's no way for you to do this. A comically

102
00:13:41.970 --> 00:13:52.980
William Cheng: Okay. So since we need to modify to this blog over here, we can we can actually write one blog to this first and then write the other one. Well, back to this later. Or we can read it in the reverse order. In that case, there are only two way to go.

103
00:13:53.910 --> 00:14:01.560
William Cheng: Okay, so, so, so, you know, so let's take a look at these two different way to see what see what will happen, what will happen over here now. So what we'll do is over here this

104
00:14:01.830 --> 00:14:07.320
William Cheng: This picture over here represent what's inside the buffer cache buffer cache to say that now I have modified to this block.

105
00:14:07.590 --> 00:14:16.500
William Cheng: This frog eventually music or through this and also the new Bronco, we can will eventually go out to the desk. More importantly, the original blog over here. We're going to have a new pointer that point to the new blah.

106
00:14:16.950 --> 00:14:23.880
William Cheng: OK, so both did a post box over here. They're modifying the buffer class of cash and now they need to be reaching out to the desk.

107
00:14:24.300 --> 00:14:25.680
William Cheng: Okay, so that this update. How's it going,

108
00:14:25.980 --> 00:14:36.420
William Cheng: It's going to get these two modification and then for some reason that this candidate has is going to write one of them to the disperse. So for example, if that this updated decide to write this one to this first what will happen right

109
00:14:36.630 --> 00:14:46.380
William Cheng: So what do I do is I will be here is that this book over here is on disk already what the first part over here is still inside a buffer cache it hasn't gone through this that this is is this point. So now what

110
00:14:46.890 --> 00:14:56.070
William Cheng: You know, so, so if everything goes okay there's no crash. There's no power outage. So in that case, the first blog over here eventually that this update has will also write it to the desk.

111
00:14:56.940 --> 00:15:02.700
William Cheng: Okay, so, sooner or later, this is going to happen. So we're going to take the stage for we here and now the file system will be in a consistent state.

112
00:15:03.000 --> 00:15:14.010
William Cheng: Okay, so any of this, of this intermediate state over here is not very consistent, but eventually is going to go. Go to the to the state over here and this is going to be happening. Most of the time, right. Most of the time you don't lose power. Most of the time you don't you the opposite.

113
00:15:15.840 --> 00:15:26.190
William Cheng: Is not gonna crash. Okay, but what if you're unlucky day if you're unlucky. It is possible that you know right right after you write the first block over here on to the desk and now you're going to end up with a crash.

114
00:15:27.000 --> 00:15:37.740
William Cheng: That. So now when you reboot the system what its gonna look like. Okay, so this blog over here, since you have a modified yet right so so inside the actual processor this blog will still look like the original state.

115
00:15:38.370 --> 00:15:46.020
William Cheng: Okay, so this way when you reboot your system. I'm going to call this one one prime over here because it doesn't look like for it doesn't like to have this like three over here.

116
00:15:46.230 --> 00:15:54.510
William Cheng: It looks like one prime, because this block over here, I see you're still sitting there exactly the way it was before the new book that you wrote on the desk. Was it turned on. Nobody's pointing to it.

117
00:15:55.860 --> 00:15:56.760
William Cheng: Okay, is this okay

118
00:15:58.290 --> 00:16:03.660
William Cheng: Okay. I mean, since this block over here. Nobody's pointing to your file system is still going to be in a consistent state.

119
00:16:04.530 --> 00:16:09.780
William Cheng: Okay, so in this case if it crashes. This time, everything actually is going to be okay. Okay, but what if you're unlucky.

120
00:16:10.020 --> 00:16:13.890
William Cheng: As it turns out that you know that this is a task will write the data out in a different order.

121
00:16:14.160 --> 00:16:20.280
William Cheng: Okay, so here's another example over here, we started out with this state over here. Right. And then we modify them in the buffer cache.

122
00:16:20.460 --> 00:16:24.510
William Cheng: And for some reason that this update has decided to write this one out to the dispersed.

123
00:16:24.720 --> 00:16:30.030
William Cheng: Guy. So now it will look like this. Right. It will write this part out to this and this will point to another blog over here.

124
00:16:30.210 --> 00:16:39.990
William Cheng: This this block currently exists, but you haven't really modify the data yet. Right. So in this case, this will be whatever garbage that it was left on the desk. Okay. And now let's say you get across right here.

125
00:16:40.350 --> 00:16:42.270
William Cheng: Okay. When you reboot the system. What are you going to get

126
00:16:42.600 --> 00:16:51.480
William Cheng: Okay, this block over here. Go to the go you guys go to the new state already so it will look just like this. But the second block over here is still have the old data and the old data could be garbage.

127
00:16:52.380 --> 00:17:01.590
William Cheng: That you're going to end up with a state like this. I'm calling this the three prime because it doesn't it doesn't really look a little as three. It's not equal to one. It's not equal to four. So, therefore, is some kind of a weird day.

128
00:17:01.830 --> 00:17:08.970
William Cheng: This is still valid pointer and now all that's happening over here is that this point to garbage. Okay, so in that case is that good.

129
00:17:09.690 --> 00:17:15.600
William Cheng: Okay, so if you think about this one is actually part of the fall. I know. So in this case, if when you open up that directory. You can see a lot of garbage.

130
00:17:16.080 --> 00:17:26.940
William Cheng: So clearly this is not a good idea. Especially if this is the route. I know if this is a route. I know you're going to end up wanting to a lot of garbage and then the original point over here that point so useful stuff. And now we don't really know what happened to it.

131
00:17:27.720 --> 00:17:35.340
William Cheng: Okay, so in this case the can be really bad because it looks at the directory become bigger and now if you go to the director entry. All you see are going to be garbage.

132
00:17:35.730 --> 00:17:41.430
William Cheng: Okay, so in this case this can be really bad, right, especially if this is the, this part of your root directory file.

133
00:17:41.970 --> 00:17:52.440
William Cheng: Then, so we need to we need to make sure that our file system is always going to be in a cousin's day. So, in the previous example, when we read the second block out to the disperse the other one is actually a preferred state.

134
00:17:52.860 --> 00:18:01.770
William Cheng: Right, so we can, if we can do that, that will then then then we're going to be reasonably happy, even though one of the this blouse or disappear, you know, from the desk. Yeah.

135
00:18:03.360 --> 00:18:09.180
William Cheng: Alright, so a more realistic example over here. So let's say that we are planning to a 10 kilowatt file.

136
00:18:09.630 --> 00:18:20.400
William Cheng: Through the fall. A. So this is the original file a original file is only 10 kilowatt the top 10 kilobytes in size. So remember, the way that we describe this is that instead of this map, they are you know 13 pointers.

137
00:18:20.610 --> 00:18:26.490
William Cheng: The first 10 points each one of them point to one kilobyte this block right so you for five is less than 10 kilobyte and says,

138
00:18:26.760 --> 00:18:35.460
William Cheng: Oh, we need other direct pointers. OK, so now we append to the file. We're going to make the fall a little bigger. So in this case, we're going to allocate the study, they love the 11 pointer over here.

139
00:18:35.640 --> 00:18:43.830
William Cheng: Because we need to point to indirect blah. So therefore, the final statements that the actual file system will look like this. The first 10 points of your still pointed original this blah.

140
00:18:44.040 --> 00:18:49.860
William Cheng: The 11th pointer over here is going to be indirect pointer, they will point to indirect blog and this one will point to the new data block over here.

141
00:18:50.790 --> 00:18:54.630
William Cheng: Okay. So in this case, you know which data block that we have to modify right we need to

142
00:18:54.900 --> 00:19:03.750
William Cheng: Modify X over here, because x over here. We're going to allocate the 11th pointer. So, therefore, that this date about this new all these other blocks will be here. There's still stay exactly the same.

143
00:19:04.500 --> 00:19:07.230
William Cheng: As they are, and now why is new and Z's new over here.

144
00:19:07.680 --> 00:19:16.140
William Cheng: Okay. So in this case, x, y, and z. And they are the dirty blondes and then we're going to, you know, release them to the update as at this update task and write them.

145
00:19:16.410 --> 00:19:21.930
William Cheng: On after this in any order that it wants. Okay. So in this case, how many different order. Can there be

146
00:19:22.440 --> 00:19:29.910
William Cheng: Okay, since we have three different dirty blocks over here the possible different combinations are three factorial right before we saw that, you know, when there's two

147
00:19:30.090 --> 00:19:32.880
William Cheng: Two factorial. The tools. That's what we have with two different combination

148
00:19:33.120 --> 00:19:47.460
William Cheng: Now we have three different ways. So what are three different ways, right, we can write X followed by why followed by Z. We also right as followed by zip all by y or y AMP z, y, z x and y, or z x, y, and z y x

149
00:19:48.210 --> 00:19:54.690
William Cheng: That. So all the six different combinations over here, maybe some of them are good. Some of them are bad, right. So in this case, you know, we sort of depends on like

150
00:19:55.080 --> 00:20:10.650
William Cheng: If we're lucky that this update has the right the spark on to this at the right time at the right place. So in that case. Every, everything that I baby. Okay. But sometimes, if we're unlucky. And we just happen to get a crash. We're going to end up with a inconsistent POS system.

151
00:20:12.270 --> 00:20:19.590
William Cheng: Right. So let's take a look at these are three different blocks over here. So for example, if we read them on this order, we're going to write x to the this first and then why did this next

152
00:20:19.800 --> 00:20:29.760
William Cheng: And then finally we're going to rise z to the desk. Right. So this guy is x will go to the new state and why we'll go through the new stay busy over here hasn't gone to the new Tuesday. And now we have a system crash.

153
00:20:30.330 --> 00:20:36.960
William Cheng: Okay. So in this case, this is very similar to the first case that I talked about. We, here we're going to have this one point to this point during this one point is this binder.

154
00:20:37.140 --> 00:20:42.720
William Cheng: And the Z data over here, since it hasn't been updated yet. So therefore, we're going to pick out whatever garbage that's left on the desk.

155
00:20:43.320 --> 00:20:48.480
William Cheng: Okay, so in this case it will look like your phone is corrupted right because over here, we take the file with grow a little bit

156
00:20:48.570 --> 00:20:57.570
William Cheng: When you go to the last block of the data. In this case, the last block of data is actually corrupted right so if he just added some, you know, maybe this is your CFC file, you know, kernels I man.

157
00:20:57.960 --> 00:21:04.680
William Cheng: You just added something to you to to do five at the end. So now, now the last block of data is gonna be complete garbage.

158
00:21:05.280 --> 00:21:12.030
William Cheng: Okay, so you're going to be pretty upset and also we open the file is gonna say that's a binary file, it's gonna be very confusing and then eventually you find out that, oh, there's some

159
00:21:12.750 --> 00:21:22.650
William Cheng: Garbage appear at the end of the fall, so it's not too bad. But again, this is, you know, if it's a regular file, it's not too bad, but it's just a directory file that this can actually look pretty bad when

160
00:21:24.480 --> 00:21:33.120
William Cheng: It is also possible that the the this update has use a different order the salary right Z out to the disperse followed by X followed by why so if after you read the data.

161
00:21:33.480 --> 00:21:46.410
William Cheng: The C block over here to this and now we get a crash. So in this case, x and y are still in the previous day. When you reboot your father we reboot your openness is that, what are you going to see while you're going to see the OH state over here. So in this case, you're going to be okay.

162
00:21:47.940 --> 00:21:56.280
William Cheng: Alright, so again you know there are six different combinations. How many of them are actually good combination. How many of them are bad combination. I just show you two of them. One of them is good. The other one is bad.

163
00:21:57.060 --> 00:22:06.390
William Cheng: So, therefore, you know, you know. So what kind of order is actually good that I'll be here, you know. So what about the other combination. So we need to make that make all the analysis to find out which way.

164
00:22:07.080 --> 00:22:15.600
William Cheng: Which way is good. Of course, it will be good if this update tasks nature can actually figure out which one's the right to the disperse. But typically, that this update has doesn't have that information.

165
00:22:15.960 --> 00:22:20.130
William Cheng: Right. What it will do is that all these pocket release that this update as they just put them together.

166
00:22:20.340 --> 00:22:29.010
William Cheng: And one thing that often do for this update has, is that what it will do is that is that when each other updated this they will try to update this using something called the elevator algorithm.

167
00:22:29.490 --> 00:22:40.800
William Cheng: Okay, so what are the elevator algorithm. The algorithm is that if you look at it, this will be here as a bunch of cylinders. So what you would do is that you will write down any sort of a sequential order according to the order of a cylinder number

168
00:22:41.460 --> 00:22:48.420
William Cheng: Okay, so this way you can minimize the see time. Will you try to write it off to the desk. Okay, so if you write data over here face on the

169
00:22:48.810 --> 00:22:55.290
William Cheng: The the the cylinder number one. In that case, you can actually end up writing all these data in all kinds of different order. Depends on where these x y&z czar.

170
00:22:56.070 --> 00:23:03.300
William Cheng: Okay. So over here, it says, does the order matter, of course, in this case the order matter because if you get a crash. I mean, if you don't get a crash in the order doesn't matter.

171
00:23:03.780 --> 00:23:10.200
William Cheng: Okay, if you get a crash. If you're unlucky. If you get a question, what then the order will matter, you will either go to a good state or you can go to a bad state.

172
00:23:11.040 --> 00:23:18.600
William Cheng: They should the order matter, of course, the answer is no, right, because we don't want the opposite. You know, we don't we don't want our forces them to depend on luck.

173
00:23:18.810 --> 00:23:25.710
William Cheng: Okay. Because if it's written in one order, then everything will work fine. And if you get a crash or and if you write it the other way. If you get a crash. Well, then in that case.

174
00:23:26.400 --> 00:23:33.300
William Cheng: You're going to either end up losing our system. Okay, so therefore it's the job of the forces them to make sure that you assist them is quite resilient.

175
00:23:34.110 --> 00:23:39.180
William Cheng: Alright, so how do you solve this problem number one over here is to say, you ask the file system, not the crash.

176
00:23:39.540 --> 00:23:45.570
William Cheng: Okay. I mean, of course, you know. Yes, you, you love nothing you lose power. Some people say, well, why don't you buy like a one of these, you know, ups.

177
00:23:45.870 --> 00:23:55.200
William Cheng: The interrupter all power supply over here you attached you attach that to a system, but eventually you're gonna run out of battery. Right, so the the the the the the ups and basically the

178
00:23:55.650 --> 00:24:03.030
William Cheng: Big batteries over here, they might lost you for a few days. But eventually, if you forgot to, you know, to, to, to replace the battery on this machine.

179
00:24:03.240 --> 00:24:10.200
William Cheng: Eventually you also run out of battery and again and again we're going to end up with exactly the same situation. Okay. So, therefore, is unrealistic.

180
00:24:10.500 --> 00:24:18.240
William Cheng: I uses and not to die. I mean, there are there are some out there though actually make sure that you have enough backup over here, you know, the battery backup. So your system will never go down.

181
00:24:18.510 --> 00:24:23.550
William Cheng: Okay, those systems are very, very expensive to build. So, so it's possible to build a system like that.

182
00:24:24.030 --> 00:24:30.900
William Cheng: So I can imagine some of the banking system are like that, right, we have a bank that has the databases that you want to make sure it never go down and in that case you will

183
00:24:31.200 --> 00:24:35.610
William Cheng: You're happy to spend a lot of money to make sure that a computer system can never go down.

184
00:24:36.060 --> 00:24:45.060
William Cheng: Okay, but again in this class. We typically talk about general purpose operating system like them on your desktop. So in that case, we're not willing to spend that much money to make sure I have it goes down.

185
00:24:45.300 --> 00:24:48.540
William Cheng: So therefore, the first approach over here we're going to consider unrealistic.

186
00:24:49.500 --> 00:24:56.460
William Cheng: But the second one over here is we're going to perform a multi step this update in an order such that this is always consistent

187
00:24:56.880 --> 00:25:08.730
William Cheng: Okay, so what we can do that, we can actually look at those order of those this the this bug over here. And then we're going to decide how to read them out to this one at a time. So this way, whenever you crash that this always going to be in a consistent state.

188
00:25:09.690 --> 00:25:19.050
William Cheng: Okay. Alright, so we've got a consistency preserving approach. The third one is that, you know, we're going to borrow some idea from the database community. They did this community, you know,

189
00:25:19.500 --> 00:25:29.850
William Cheng: If you haven't taken the database class, you probably learned a lot about something called a transaction. Okay. If you're running transaction. You can guarantee that your database system is going to be in a consistent day no matter, you have a crash or not.

190
00:25:30.390 --> 00:25:35.820
William Cheng: Okay, so we're going to take that idea that's used by the database is there and we're going to use it to implement the file system. So, in that case.

191
00:25:36.030 --> 00:25:44.670
William Cheng: Inside of us that will also run run transactions. So once you start running transaction that no matter what the way you will your system crash your system will always be in a consistent state.

192
00:25:45.240 --> 00:25:55.680
William Cheng: Okay, so we're going to take a look at these two approaches, I first want to take a look at the consistency preserving approach. And then, so maybe in the third part of today's lecture, we're going to look at how to run transactions. Right.

193
00:25:58.980 --> 00:26:08.910
William Cheng: So here's sort of the picture that shows you the relative, you know, relatively you know how good they are on the horizontal axis. Here's the consistently consistently consistency access

194
00:26:09.330 --> 00:26:16.560
William Cheng: The more to the right, the more consistent your fastest them as the vertical axis of years gonna be performance. The higher you are, the better performance you are

195
00:26:16.980 --> 00:26:27.360
William Cheng: So over here, it says this is the part of us is that the five forces that they have the same consistency characteristic. They're all not very consistent, right, in case you get a crash. Right. So again, if you don't get a crash. Everything's fine.

196
00:26:27.660 --> 00:26:30.750
William Cheng: When you get a crash. That's what Beth is going to happen. Okay, so

197
00:26:31.140 --> 00:26:39.420
William Cheng: So when you get to crush, then this is the VA system and the finalists, and then they're just as bad as each other, even though the fast so that has much better performance than the system level system.

198
00:26:39.750 --> 00:26:44.610
William Cheng: That and then comes the self update. So I saw updates over here is the one that we just mentioned over here.

199
00:26:45.000 --> 00:26:51.840
William Cheng: So, this one has better consistency. But again, there are not as good as the running transactions. Okay, so first we're going to see. I see this

200
00:26:52.290 --> 00:27:02.220
William Cheng: Particular approach called soft, soft update and to see how they actually improve improve consistency there. And then on the third part we're going to look at the transactional approach.

201
00:27:04.560 --> 00:27:15.360
William Cheng: So the main idea because when we write to the days when a writer, very, very carefully, we're going to write the data out onto this in an order such that no matter when we get a crash the file system will always be in a consistent state.

202
00:27:15.960 --> 00:27:20.400
William Cheng: Okay, so, so how do we actually get guarantee that Raza over here the tricks is over here is that

203
00:27:20.820 --> 00:27:29.550
William Cheng: We're not going to release all the this blog to this up a task because once we release it. That is the data to this up a task that this update has is going to run the elevator were there and then it's going to update

204
00:27:30.150 --> 00:27:33.450
William Cheng: This part in the wrong order. Okay, so what we're gonna do is we're gonna

205
00:27:35.700 --> 00:27:39.780
William Cheng: We're going to basically use the combination of right back and right through algorithm.

206
00:27:40.380 --> 00:27:49.230
William Cheng: Okay, so that we're gonna have some fun. We're going to use it right through our there to to to perform a synchronous right to the day so again synchronous over here doesn't mean that they think the right operation is not finish.

207
00:27:49.800 --> 00:27:59.790
William Cheng: The right version doesn't return until the operations finished. Okay, so therefore we perform a synchronous right we need to wait for the right to get on to the desk when the data go on to this, then the writers finish.

208
00:28:00.690 --> 00:28:12.030
William Cheng: Okay, so this is going to impact the performance because you are not allowing that this update has to use the elevator. Our that so overall this is going to be slower. Right. But in this case, again, no matter where you crash your system will be in a consistent state.

209
00:28:12.540 --> 00:28:19.140
William Cheng: That right alright so the different way to go. So what does it do a synchronous right or you can actually, you know, so what you can do. They can also

210
00:28:19.380 --> 00:28:28.230
William Cheng: Go, the more complicated away. Whenever we modify the data. We're going to actually build a data structure to describe the teeth dependency of all these that this blog over year. And then we're going to handle

211
00:28:28.710 --> 00:28:36.120
William Cheng: That we're going to take this dependency data structure. We're going to give that this update has to say hey will you write it out to the desk. Make sure that you write it out with this.

212
00:28:37.050 --> 00:28:42.450
William Cheng: In this kind of order. Okay. But in that case, he will. He will actually it's quite complicated.

213
00:28:42.990 --> 00:28:48.900
William Cheng: So, so typically people, you know, I guess, I guess in the 80s when people try these approach, they tend not to do something like that.

214
00:28:49.230 --> 00:28:58.650
William Cheng: Okay, so that basically they they basically perform synchronous right and then in a combination of rice to and right back. Whereas, let's take a look at a good example to see how this is done.

215
00:28:59.490 --> 00:29:02.010
William Cheng: Okay, so let's say that we go through our original example.

216
00:29:02.220 --> 00:29:12.450
William Cheng: What we need to do is that we need to start with it, you know. So with this particular state. And then when we need to do is that we need to modify the note over here, we need to allocate a new know and then modify know as we had appointed a new note.

217
00:29:12.960 --> 00:29:23.280
William Cheng: Okay, so this is basically the original example right we we we start with the with the know a over here. Now we're going to modify modify a and also we create a new note over here for a two pointer.

218
00:29:24.690 --> 00:29:31.680
William Cheng: Okay, so in this case we're going to go from this data. When do we start with this day, we need to go to a new state. So the question is, again, we need to perform to describe

219
00:29:31.890 --> 00:29:37.350
William Cheng: Wild for Brock and the other one for block C. So if you are using software update. What would you do

220
00:29:37.770 --> 00:29:47.040
William Cheng: Okay, so since this this blog over your depends on the new wine. So what we're gonna do is, I'm going to write the one that has the least amount of dependency first right onto the days using right through.

221
00:29:47.820 --> 00:29:52.800
William Cheng: Guys, and this is what we're gonna do is, I'm going to write notes at the block see over here on to the desk over here.

222
00:29:53.010 --> 00:30:02.040
William Cheng: Using a synchronous right over years. Okay, what happened is that he's had actual process that we're gonna perform the right and then we're not gonna, we're not going to go back to what we're doing. Until this right has

223
00:30:02.730 --> 00:30:04.710
William Cheng: Has been verified to have gone on to the desk.

224
00:30:05.190 --> 00:30:11.790
William Cheng: There. So what is this data has been verified has gone on to the this, then we can actually release block a over here, which is the new block.

225
00:30:11.970 --> 00:30:20.640
William Cheng: Over here. So this block of your nose dirty, we can release it asynchronously to that this apostasy. Now this book. You can read it to the this anytime you want.

226
00:30:21.540 --> 00:30:26.820
William Cheng: Okay, if we don't get a at this crash. So eventually this this blog is going to go on to this and now we're going to the new say

227
00:30:27.150 --> 00:30:32.970
William Cheng: What if you get this crash. Okay. If you go to this if you get it this class or a necklace won't go back to the previous day, then

228
00:30:33.240 --> 00:30:38.940
William Cheng: We're going to realize that we're going to go. I will go back to the previous state over here because block a has not been modified

229
00:30:39.900 --> 00:30:50.580
William Cheng: That. So therefore, the pistol will look like this. So if we get a crash over here blocks. He has one on to the desk and now the asynchronous right that this didn't get finished with the crash. So no

230
00:30:51.390 --> 00:30:55.830
William Cheng: No date over here still pointing to the OH NO be over here. So therefore, the picture would look like this.

231
00:30:56.790 --> 00:31:08.280
William Cheng: Guy the future look like this. Right. If you get a crash, and now we recover and now no seat over here is nowhere to be found. Because this why nobody. There's no point of pointing to this one. So this one is going to be lost inside of our system.

232
00:31:09.120 --> 00:31:18.990
William Cheng: Okay, so where's proxy. Now the blocks. He doesn't belong to a free list doesn't belong to the file system hierarchy. So, you know, well you know you'll fall assist them your entire days now. And now one block has disappeared.

233
00:31:20.400 --> 00:31:28.860
William Cheng: Okay, so if this happens all over and over again. You are. This is gonna get smaller and smaller and smaller us at some point in her you you that this is the you

234
00:31:29.190 --> 00:31:32.550
William Cheng: A lot of a lot of space on your desk just kind of disappeared into thin air.

235
00:31:33.060 --> 00:31:42.570
William Cheng: Okay, so in that case what we need to do is that inside a colonel, we need to run another test that will actually discover these these this block and what it will do is that they watch you put it somewhere. Okay.

236
00:31:43.020 --> 00:31:49.650
William Cheng: So this guy is what it will do is that we can actually take this this blog over here and then we can actually put it to trash, we can return it back to the free list.

237
00:31:51.060 --> 00:31:55.830
William Cheng: Okay, so, so, so, so, so, so, so that we bought way to do it.

238
00:31:56.070 --> 00:32:05.130
William Cheng: As it turns out, for the fast, fast faster. So what it will do is that instead of returning to the free list. What it will do is that they will actually run a sort of a thread inside of Colonel called the disk.

239
00:32:05.460 --> 00:32:07.890
William Cheng: I don't know if that that's what the terminology, they use in the past was is that

240
00:32:08.640 --> 00:32:13.440
William Cheng: So there's some system, they'll call that this scavenger threat. Okay, that this scavenger. What it will do is

241
00:32:13.740 --> 00:32:23.430
William Cheng: It will go to this look at every blog and for every blog, they would determine whether that block belong to the free list or if you want to the file, file system hierarchy. If you don't do the free list or the foster

242
00:32:23.820 --> 00:32:33.150
William Cheng: System hierarchy, then in that case it will not touch that blog. But if you find a blog that doesn't belong to the free list and it doesn't belong to the file system hierarchy is, what does that mean that means that it's a block like this.

243
00:32:33.870 --> 00:32:41.850
William Cheng: That. So, in that case what he would do is that you actually take this this blah, create a file just with this this book and he will add it to a directory that the Lord has called

244
00:32:42.210 --> 00:32:49.200
William Cheng: Lost and Found inside your processor. So if you go to the roof us is that there's a directory called Lost and Found where they have a lot of files like this.

245
00:32:49.590 --> 00:32:54.750
William Cheng: Okay, what do they do is they will also send the user email to say that, hey, you know, when you know what the discs.

246
00:32:55.050 --> 00:33:02.040
William Cheng: A scavenger you know thread, you'll try to look for these, the, the, the, the, the, this, this bar it found one of them actually belongs to you.

247
00:33:02.520 --> 00:33:11.280
William Cheng: So now I have created a file that's a loss of our maybe, you know, maybe you can actually recover data format that. So in that case, if it turns out that this file is one of your see file.

248
00:33:11.550 --> 00:33:19.860
William Cheng: Right, this will be a purely texts out what you would do that, you will get the email you go to sideline lot slash loss info you can open this file, and you can actually copy and paste that fall into your

249
00:33:20.040 --> 00:33:28.920
William Cheng: Into the code that you're writing. So this case you want to recover from this particular error. Okay. If it turns out that this is one of the data blog in a binary file or not. Guys, of course, you're out of luck.

250
00:33:29.880 --> 00:33:37.890
William Cheng: Now, so, so in the past is, is there any way to use this pretty good code approach. So at some point as fast. Most of them, they actually they implement the soft update approach over here.

251
00:33:38.070 --> 00:33:48.150
William Cheng: And then in this case, this kind of inconsistency over here. I mean this, you know, this note disappear from, you know, disappear from the desk. So in this case, it creates an inconsistency inside your foster said

252
00:33:48.360 --> 00:33:53.310
William Cheng: But this kind of inconsistent is considered innocuous in consistencies okay because it doesn't really

253
00:33:53.760 --> 00:34:03.000
William Cheng: Doesn't really hard to foster said, The only problem is that your this is getting smaller and smaller and smaller than. So therefore, in the SOF update innocuous inconsistency is considered acceptable.

254
00:34:03.690 --> 00:34:17.370
William Cheng: Okay, so this one will you be your fastest the fastest misspelling the previous date, even though. Now you have lost the block. Okay, so, so, so again, you know, some people actually implement this approach and and you know that people are actually reasonably happy with this pretty good approach.

255
00:34:19.440 --> 00:34:28.020
William Cheng: And actually when you're installing Ubuntu how the feeling so been to onto your Otter raw hardware and not inside of inside a virtual machine.

256
00:34:28.710 --> 00:34:37.080
William Cheng: One thing that you can do that. You can actually configure your fastest them and their weapons that will you try to install Ubuntu, it will ask you what kind of file system. You want one of them is called

257
00:34:38.010 --> 00:34:48.660
William Cheng: Right, so typically these are your choices are you 6263 X 34. So these are your actual files or Sam today. You know, I guess if you're running Ubuntu 16.04 they are using yesterday for

258
00:34:48.930 --> 00:34:54.120
William Cheng: That in the grill DAY OF THE ST to, as it turns out, UX team to is that FFS clone.

259
00:34:54.540 --> 00:35:02.640
William Cheng: Okay, it's a clone of the fast processor. So if you install a file system like EMC to you will notice that in the root directory. There's a directory costs loss and file.

260
00:35:03.180 --> 00:35:09.630
William Cheng: Okay, so therefore, whenever you see that director, you say, ah, you are actually implement self update and that's what you end up with this particular directory. Right.

261
00:35:10.230 --> 00:35:18.390
William Cheng: Right. I think today if you use txt for yesterday for actually running transactions is that apostles them. So, therefore, there's no loss and Tom directory. Yeah.

262
00:35:21.540 --> 00:35:29.010
William Cheng: So, so let's go to our example over here, right, if, if x, y, and z over here are dirty blocks. So in that case, what will be the right order to write to the desk.

263
00:35:29.460 --> 00:35:37.620
William Cheng: Okay, so again we need to write on to the data to the idea is to have the least amount of dependency. So in this case, Z doesn't depend on anything. So therefore, we will rise the to the disperse

264
00:35:37.950 --> 00:35:42.330
William Cheng: Go. So again, we need to write this data synchronously to the desk right so we need to wait for it together.

265
00:35:42.510 --> 00:35:49.920
William Cheng: And then who do we write next or we need to. Why, why are we here because why depends on Z and X depends on both y and z. Right, so x has the most amount of dependency.

266
00:35:50.100 --> 00:35:57.450
William Cheng: And why has the second most amount of dependency. Right. So after we raised the synchronous to this and now we need to write why synchronous

267
00:35:57.690 --> 00:36:04.350
William Cheng: Let's do this. And then eventually when we're done over here, we can release this blog X over here to this up a task that

268
00:36:04.920 --> 00:36:11.790
William Cheng: So in that case, you know. So in this case, you know, when you crash. You can only crush either before he went on to the desk or between

269
00:36:12.300 --> 00:36:21.540
William Cheng: Or after he went on to the desk or MP4 why went on to this or, you know, after we went out to the desert and before long to this or you can get a crash after X went on to the desk.

270
00:36:22.380 --> 00:36:27.390
William Cheng: Yeah. So in this case, if you crash to earlier, we hear this guy's. Will you read blue the bosses and what will you see

271
00:36:27.600 --> 00:36:32.310
William Cheng: While you will see the old state of the file system, which is a consistent stay for your insight, your pastor, Sam.

272
00:36:32.520 --> 00:36:41.340
William Cheng: You can also crash after you rise the synchronously onto the desk and it's going to look like this right again when you reboot the file system, you will go to a previous day, and now

273
00:36:41.640 --> 00:36:45.420
William Cheng: Z brought over here. It doesn't belong to the file system hierarchy and visible on the free list.

274
00:36:45.660 --> 00:36:53.760
William Cheng: Again, that this scavenger a thread inside of Colonel will have the discovered this, you know that this blog and then return to return it into the Lost and Found directory now.

275
00:36:54.360 --> 00:37:06.840
William Cheng: What if you get a cross between one and x over here. So again, y and z over here. They both disappear from your desk right and then we reboot the file system is still go to the previous day, which is the which a good day again with innocuous and consistency.

276
00:37:07.290 --> 00:37:19.380
William Cheng: So in this case is still got so bad that finally if you get a crash after the block X over here has written out to the display. In this case, when you reboot, you're going to end up with a new state of houses me also is going to be in a consistent state.

277
00:37:20.400 --> 00:37:25.110
William Cheng: Okay, so it looks like if you take this approach, analyze these data and you synchronous right to the

278
00:37:25.650 --> 00:37:29.820
William Cheng: To the, to the desk for all these data blocks over here has the least amount of dependency.

279
00:37:30.000 --> 00:37:40.950
William Cheng: And then you can finally released the final blog, you know, to the display task. So in this case, no matter when you crash. You're always gonna get we're gonna get into a consensus day or a state with innocuous in consistencies

280
00:37:41.820 --> 00:37:51.270
William Cheng: Okay. So in this case, you know, it's always good. Right. But although this is slow because they're required to synchronous right you know for y and z over here. And then, you know,

281
00:37:52.890 --> 00:38:05.520
William Cheng: Yeah, so, so, so in this case, what happened is that we are we are slowing down the file system for something that rarely happens right because the failure, you know, condition when you lose power we were putting aside offices that that pretty much never happens.

282
00:38:05.940 --> 00:38:16.890
William Cheng: Okay. But in this case, we're so paranoid. We actually we're going to slow down the process them for things that never happened. So this case, you know, even though it's a solution. It's really not a very good solution. Okay.

283
00:38:18.420 --> 00:38:24.300
William Cheng: All right, let's take a look at another example we're going to create a new file with one day to blog over here. So in the beginning over here.

284
00:38:24.660 --> 00:38:29.070
William Cheng: We create a new file. We're going to add it to the directory. I know. So instead of directly. I know.

285
00:38:29.340 --> 00:38:36.390
William Cheng: The you know the direction file has to be changed and also would create a new file, the funny stuff. I know they also need the data blog over here.

286
00:38:36.780 --> 00:38:42.300
William Cheng: So in this guy is going to end up with the same picture X depends on why why depends on z x, y, z or dirty boss. Right.

287
00:38:42.450 --> 00:38:54.330
William Cheng: So, so we already know what the solutions are so again we wish you right Z to this first, followed by why. And then, and that at the end, we can release block x to the this update has and that can be reached out to this anytime he wants because

288
00:38:54.900 --> 00:38:58.140
William Cheng: This solution will work. But again, you know, it's not very efficient.

289
00:38:59.910 --> 00:39:07.170
William Cheng: The next example over here, we tried to move a file, we tried to move this fall from directory extra directory why. Okay, so here's a directory or directory

290
00:39:07.860 --> 00:39:13.800
William Cheng: You are doing Colonel to, you know, how do you implement this right the way you implement this, and that you need to unleash this fall from the original directory

291
00:39:13.950 --> 00:39:22.590
William Cheng: And then add another link and the second directory. Right. There's no atomic operation to say move a file for one directory to another directory. So again, need to divide it into two operations over here.

292
00:39:23.160 --> 00:39:29.430
William Cheng: The final say is going to be like this or avoid this file, I note over here. If you don't do another directory or so in this case which blocks are dirty.

293
00:39:29.790 --> 00:39:40.140
William Cheng: Okay, so only the x and y over here. These are dirty, because x over, you need to you need to, we need to remove a link and why over here, you need to Adelaide so excellent wide over here are the only blocks that are dirty.

294
00:39:40.620 --> 00:39:51.900
William Cheng: Okay, so this guy, we only have two blocks are dirty. There are two ways to write them out to the desk. We can do it. The first way on the second way. So, so we can write x prime to this, I followed by white pride or we can write why private through this eyeball by X right

295
00:39:53.040 --> 00:40:00.900
William Cheng: Okay. So as it turns out. So in this case, let's analyze as well happens right if we write x prime over here to the desk and then we are white pride. What if we get a crushing them at all.

296
00:40:01.830 --> 00:40:11.040
William Cheng: That. So if you look at this picture over here x, you know, x is going to go to Newsday be like this. And why is going to stay in the states are going to look like this. So in this case, this fall will disappear.

297
00:40:11.730 --> 00:40:17.130
William Cheng: Okay. So in this case, you know, when you reboot the system. This particular file is not in Directory Access nine directory why

298
00:40:17.490 --> 00:40:22.080
William Cheng: Okay. So clearly this is really bad, right. So, so again this is no good is not a good solution.

299
00:40:22.290 --> 00:40:30.270
William Cheng: What if we take the second solution over here, right, we're gonna right why problem out to the disperse. And now we have a crash. So in this case, the word problems going to go to the new new state.

300
00:40:30.480 --> 00:40:42.630
William Cheng: Right, so y prime is going to go into the new stay an extra stay to stay. So when we saw, and then we're going to crash over here. Well we reboot the system over here, this particular file is reachable in both, you know, directly ask them directly. Why

301
00:40:43.350 --> 00:40:46.200
William Cheng: Okay, so in this case is that innocuous inconsistency.

302
00:40:47.790 --> 00:40:54.630
William Cheng: I mean, it sounds like it doesn't really hurt anything. But in this case, this could be a security risk that maybe the reason you're trying to move far

303
00:40:54.840 --> 00:41:03.120
William Cheng: You know this telephone directory X to directly why maybe directory X is accessible by everybody inside your system and you try to move into a private directory

304
00:41:03.300 --> 00:41:10.980
William Cheng: Right. So in this case, you're going to assume that the data was successfully moving to a private directory. As it turns out, you know, people in a public place, they can still access your file.

305
00:41:11.280 --> 00:41:17.760
William Cheng: Okay, so now this become a director a security risk. So therefore you cannot claim that this is innocuous inconsistency.

306
00:41:19.110 --> 00:41:24.780
William Cheng: Alright, so you can see that a software update doesn't really give you a good consistency, because there are these cases where there's nothing you can do.

307
00:41:25.800 --> 00:41:32.280
William Cheng: Alright. And the reason I did that the software has nothing you can do. Is that because there's no dependencies between x and y.

308
00:41:33.000 --> 00:41:37.050
William Cheng: Okay and this example over here, right, we move this file from from the directory extra directory why

309
00:41:37.200 --> 00:41:47.070
William Cheng: So when we try to analyze the dependency for x and y. There's absolutely no dependency. So therefore, the you know the the forces them to decide to they can write these data back after this anyway was

310
00:41:47.250 --> 00:41:57.750
William Cheng: As it turns out, no matter which way you the right to them out to the to the desk and none of them is actually a good option. And again, in case you you crash, then it's going to be bad if you don't crash everything so gonna be okay.

311
00:41:58.800 --> 00:42:06.270
William Cheng: Yeah, so, so, so that there's no solution for this. So, therefore, some updates has doesn't really have very, very good consistency. Right.

312
00:42:08.070 --> 00:42:16.080
William Cheng: Oh, sorry reality. There's also more complication over here. I'm going to take a look at the example of your this is very similar to example to over here.

313
00:42:16.260 --> 00:42:23.100
William Cheng: We have a fall, we have a directory. I know, and it will fall I note, I'll be here with you create a new this blog, you know, for for that particular file. Okay.

314
00:42:23.880 --> 00:42:30.060
William Cheng: So, so, so, so you're getting this case over here at the file and the data block with this file, they are brand new.

315
00:42:30.480 --> 00:42:42.240
William Cheng: That if it turns out that they were modifying existing blog that this problem is going to be a lot more complicated, right. So let's take the simple example over here. Where did they did the you know the the fall, I note and the file data blog over here.

316
00:42:42.630 --> 00:42:50.520
William Cheng: They are actually you know they they over here. There's actually put onto a file. I know. So in this case, these are the two two new notes over here.

317
00:42:51.240 --> 00:42:59.130
William Cheng: So again, this is x, this is why this is z where we do our analysis, we will rise see over here to this synchronously, followed by one followed by x

318
00:42:59.580 --> 00:43:05.100
William Cheng: Okay. So this example is a little more complicated. The over here. These two things that are both I knows and we see that the

319
00:43:05.400 --> 00:43:15.930
William Cheng: Data structure are very, very small. So in reality, you know, over here can actually combine them into the same disbar, it is possible. So yeah, for the I know the I know exists inside the is the is

320
00:43:17.430 --> 00:43:28.920
William Cheng: The is you can put many, many I notes inside of this block. So it is possible that you know that these two this bug of your actually belong to exactly the same. This bar that. So in that case, the picture look like this.

321
00:43:30.360 --> 00:43:42.450
William Cheng: Okay, so again this is x over here. This is why, and then z. Turns out, z is equal to x. Okay. So, therefore, x depends on why why depends on x. So in this case, this is called circular dependencies. So when you try to, you know, so

322
00:43:43.770 --> 00:43:49.290
William Cheng: So, so in this case with your circular dependency, how do you analyze this and in order for you to figure out how to write it out to the test.

323
00:43:49.800 --> 00:43:51.360
William Cheng: Okay, there's really no way to do this.

324
00:43:51.870 --> 00:43:57.570
William Cheng: So over here, there's a little note on the previous slide over here. It's just say, well, you try to analyze it dependency of all these this blog over here.

325
00:43:57.750 --> 00:44:01.320
William Cheng: You can run an algorithm. So if you have taken CSI 70 you will learn and sell them.

326
00:44:01.530 --> 00:44:10.110
William Cheng: Know it's topological sore from logical source is going to give you a linear order so that way you tried to write it out in that linear order all the dependencies over here will be satisfied.

327
00:44:10.500 --> 00:44:14.550
William Cheng: Okay. The only thing that break topological sore is a circular dependency.

328
00:44:14.880 --> 00:44:22.860
William Cheng: Of so we have circular dependency over here, like this one, when you try to run the logical sort of lego store is going to give off the said there's no way I can generate a linear order.

329
00:44:23.400 --> 00:44:29.040
William Cheng: So that you know that that dependency is not gonna be violated. Yeah. Alright.

330
00:44:29.670 --> 00:44:34.350
William Cheng: So this is the primary problem with the soft, soft day is that we're going to end up with a circular dependency.

331
00:44:34.710 --> 00:44:43.920
William Cheng: Okay, so in in a simple case like the case that we talked about. We here where this. I know this. I know is new and the data blog over here consists of a puncture the point of the I know

332
00:44:44.130 --> 00:44:50.370
William Cheng: You know, this is also new. In that case, you know, the, the, the people who try to write the fastest and they really want to get this to work.

333
00:44:50.670 --> 00:44:54.060
William Cheng: Okay, they actually come up with a solution for a Special K like this.

334
00:44:54.480 --> 00:45:04.110
William Cheng: Okay, so let's see what are the solutions over here, the solutions over here is that we're going to take this this blog over here without modifying the, oh this blog over here. We're only modify the bottom part over here.

335
00:45:05.280 --> 00:45:15.120
William Cheng: Okay, so now you know this will be the entire this Bob. We only modified the bottom part and then we're going to perform a synchronous right to to the desk. Okay, so in this case we get a crash, what will happen.

336
00:45:15.780 --> 00:45:23.520
William Cheng: What if we get a crash raptor whenever we turn over here, we raise him back over here the directory. I know that we're still going to be in the States, so therefore it will not pointed this

337
00:45:24.180 --> 00:45:26.250
William Cheng: Guy. So therefore, we're going to be in the consistency day right

338
00:45:26.490 --> 00:45:31.890
William Cheng: Alright, so if we don't get across. The second thing we're going to do that we're going to write this thing that blog of us again this is x and y over here.

339
00:45:32.070 --> 00:45:39.450
William Cheng: We're going to run wild over here to the dis synchronously. Again, if we get a crash was going to have a we're going to go back to the state over here because this pointer is still pointed the old data.

340
00:45:39.750 --> 00:45:49.680
William Cheng: Okay. And finally, we're going to update the first part of this data ball right here with the new information that we're modifying and. Now again, this one go to the new state.

341
00:45:50.730 --> 00:45:55.680
William Cheng: Block X over here. And then finally we're going to release this brought out to the discipline. So glad. So, this can be

342
00:45:55.950 --> 00:46:06.660
William Cheng: written out to the desert anytime. Right. So again, if we get a crash before this went out to this, we still go back to the previous day, because the data that you modify over here is simply, is that a buffer cache and not inside the actual file system.

343
00:46:06.990 --> 00:46:12.270
William Cheng: Okay, if it turns out that you get a crash after you read this did a blog I thought of this, then in that case it will go to the new state.

344
00:46:12.990 --> 00:46:25.320
William Cheng: OK. So again, over here, this is slow because we have, you know, we have to synchronous right and then followed by a regular right. OK. So again, most of the time passion doesn't happen, but we're paying for all the time. Every time we have circular dependency.

345
00:46:26.040 --> 00:46:36.180
William Cheng: Right. Alright, so, so I guess people in the you know the 90s. They really try to get this to work. So, so, so the software update is implementing the fast also say in 1994

346
00:46:36.840 --> 00:46:40.500
William Cheng: So you can actually see that, you know, whenever you you know that this is actually

347
00:46:41.280 --> 00:46:53.490
William Cheng: In the end is slow down the file system and also there are cases like when you try to move a file for one director to another. In that case, where you get a crash, you know. The Fosters and we'll go into the cocoa going to fascinate yeah

348
00:46:54.990 --> 00:46:56.220
William Cheng: Alright, so

349
00:46:57.300 --> 00:47:04.620
William Cheng: The next thing we're going to look at this transaction. So there's actually a good breaking point. So next time we're going to see how to run transaction instead of foster Sam. So today.

350
00:47:05.130 --> 00:47:13.740
William Cheng: Pretty much all the file system that you see today they're running transactions. Okay, so it's a long time ago when they would discover as well when they're implementing so instead of database community.

351
00:47:14.310 --> 00:47:27.240
William Cheng: They will consider a kind of a very slow operation. So they're all sort of people figure out how to do this in an efficient way. So, so, and then operating system started yesterday so running transactions. So what I'll take a look at, take a look at a prison.