tag: subject/distributed-system

TIL htop C100k problem Timer interrupt(A periodic interrupt that allows the OS to stop the current thread’s execution and schedule a different thread) Coroutine Notes 1-thread and n-thread: Advantages 1 Thread n Threads Memory/Performance Fairness in scheduling No locking Easier to understand and write the code Disadvantages 1 Thread n Threads Stack Ripping Locking Debugging hard Hard on performance Fibre vs Coroutine vs Async: Fibre/Coroutines have their own stacks(benefits like stack trace/shared variables)....

File systems that already existed during GFS - GPFS/Lustre/AFS Why GFS was built instead of using existing ones? GPFS and Lustre are paid systems(could be the main reason) Specialized workflows Most common solutions to problems that come up: Batching Pipelining Separation of control and data flow Consistency Weakly consistent Mitigates consistency maintenance to its applications Takeaway: Satisfying the need is the main goal and to have that industries try to create simple system designs....

TIL: POSIX Reason for MapReduce(Why use a distributed system?) Lots of data(1 PetaByte), machines had 160 GB storage so can’t process in one machine I/O speed is very low(performance) Fault tolerance(Tolerate machine and disk failures) Application programmers don’t need to work on systems and making sure their job is running in a distributed fashion Workflow Input Map Reduce Result key, value K1 <v1, v1`, v1`` > K1 R(v1, v1`, …) K2, v2 K3, v3 Examples Word Count Sort Reverse Links(Used for page ranking) Input - key: URL, value: HTML Map - (<target1, src1>, <target2, src2>,…....