Resources

Requisite

Introduction

What is an operating system?

The operating system is the Computer land’s government.

What do operating systems do?

Operating system tasks are managing processor, memory, device, storage, and providing an application interface.

Story

A brief story from the 1950s to the present.

Mainframes.

No operating systems?

https://www.youtube.com/watch?v=-rPPqm44xLs

https://www.youtube.com/watch?v=3rVfUbDsdf8

First operating system

The first industrial operating system released was GM-NAA I/O, sometimes called GMOS, of General Motors and North American Aviation for the IBM 704 computer in the May of 1956. Its purpose was batch processing.

However, the first operating system was released in 1953 for EDVAC.

Ryckman, George F. (April–June 1983). "17. The IBM 701 Computer at the General Motors Research Laboratories". Annals of the History of Computing. IEEE. 5 (2): 210–212. doi:10.1109/MAHC.1983.10026.

G. W. Reitwiesner, "The first operating system for the EDVAC," in IEEE Annals of the History of Computing, vol. 19, no. 1, pp. 55-59, Jan.-March 1997, doi: 10.1109/85.560738.

Batch processing systems

Batch processing systems are one that runs programs with no user interaction.

They are ongoing in scripts such as batch files (.ps) in Windows-based or bash files (.sh) in Unix-based. For instance, assuming a file with thousands of users, a possible need is to send emails to them every day at 6 am with an ad.

Notes

IBM Corporation. "What is batch processing?". zOS Concepts. Retrieved Oct 10, 2019.

Multiprogramming

Time-sharing

Multitasking

UNIX

Dennis Ritchie, Brian Kernighan, Ken Thompson, and others in Bell Labs released Unix in November of 1971.

Notes

Ritchie, Dennis, and Ken Thompson, “The UNIX Time-Sharing System,”, Communications of the ACM, 17,1 (July 1974), 1897-1920

Unix: A History and a Memoir. Brian Kernighan

Youtube Unix intro
The Evolution of the Unix Time-sharing System
, Dennis M. Ritchie, 1979.

Minicomputers

Multiprocessor systems

Virtualization

Emulation

Hypervisors

https://www.muylinux.com/2014/05/28/exirion-os-so-libre-mexico/

Specialized OS

https://developer.nvidia.com/drive/driveos

Today

Some popular distros are Ubuntu, CentOs, Fedora, Debian, and Red Hat Enterprise Linux. By the way, I use Arch Linux.

Notes

Just for Fun. The Story of Accidental Revolutionary. Linux Torvalds and David Diamond.

TempleOs

Today

[1] Samsung U.S. Newsroom, “Six Advantages of Tizen OS on Samsung Smart TVs,” Samsung US Newsroom, Jun. 15, 2021. [Online]. Available: https://news.samsung.com/us/six-advantages-of-tizen-os-on-samsung-smart-tvs/. [Accessed: Aug. 10, 2022]

[2] Trinder, Garry. "The Xbox Operating System". Xbox Engineering. MSDN. Archived from the original on November 2, 2018. Retrieved October 31, 2018.

[3] "Console Security - Switch". 30 December 2017. Retrieved 30 May 2018.

[4] Patel, N. (June 7, 2010). "iPhone OS 4 renamed iOS 4, launching June 21 with 1500 new features". Engadget.

[5] “EMUI 10.1 - HUAWEI Global,” Huawei.com, 2020. [Online]. Available: https://consumer.huawei.com/en/emui10-1/. [Accessed: Aug. 10, 2022]

[6] “PS5 System software (US),” Playstation.com, 2022. [Online]. Available: https://www.playstation.com/en-us/support/hardware/ps5/system-software/. [Accessed: Aug. 10, 2022]

Programming environment

https://wiki.osdev.org/How_kernel,_compiler,_and_C_library_work_together

API (Application Programming Interface)

“API” is a mechanism that enables two or more software components to communicate with each other with a specification hiding the internal details [1]. The simplest example of API is a client and server, where the server shows its headers, and the client calls them.

For instance, Amazon renders its amazon.com with Web APIs:

[1] “What is an API? - API Beginner’s Guide - AWS,” Amazon Web Services, Inc., 2022. [Online]. Available: https://aws.amazon.com/what-is/api/. [Accessed: Aug. 10, 2022]

POSIX

POSIX is an API chiefly UNIX-like and it stands for Portable Operating System Interface.

IEEE Standard 1003.1

C++ https://www.iso.org/standard/79358.html

C https://www.iso.org/standard/74528.html

Doug McIlroy, Elliot Pinson, and Berk Tague described Unix philosophy as follows:

• Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features.”

• Expect the output of every program to become the input to another, as yet unknown, program.

Windows systems

https://docs.microsoft.com/en-us/windows/win32/

Windows, POSIX equivalence

FORK, spawn …

Operating system

Operating System provides an API to processes, memory, filesystem, I/O devices, storage, network, users, and shell since it manages hardware resources.

For instance, printf in C, it’s provided by stdio.h-—UNIX headers to the standard I/O [2].

[2] Kernighan, Brian; Ritchie, Dennis M. (March 1988). The C Programming Language (2nd ed.). Englewood Cliffs, NJ: Prentice Hall. ISBN 0-13-110362-8.

The operating system zoo (Classification of Operating Systems)

Embedded operating systems

Real-time operating systems

Operating systems distinguish by their key parameters: time. In other words, they focus on doing tasks by a deadline. Examples are nuclear reactor systems, automobile manufacturing assembly line systems, military drone systems, or ballistic missile systems.

Instructional operating system and kernels

PINTOS

Pfaff, B., Romano, A., & Back, G. (2009). The pintos instructional operating system kernel. ACM SIGCSE Bulletin. Association for Computing Machinery. doi: 10.1145/1508865.1509023

"Xv6, a simple Unix-like teaching operating system". Retrieved 2014-09-22. Xv6 is a teaching operating system developed in the summer of 2006

MINIX

OS/161

System/161 2.x Software and Hardware Manual

David A. Holland, Ada T. Lim, and Margo I. Seltzer, A New Instructional Operating System. Proceedings of the 2002 ACM SIGCSE Conference on Computer Science Education, March 2002.

PDF

Research

https://dl.acm.org/sig/sigops

https://ieeexplore.ieee.org/search/searchresult.jsp?newsearch=true&contentType=periodicals&queryText=operating systems

https://cacm.acm.org/magazines/2022/5/260363-technical-perspective-what-is-the-ideal-operating-system/fulltext

Ecosystem

CISCO

RedHat

Jobs as Sysadmin,

X operating system contributor where X is Linux, Windows, Android, and so on.

HelpDesk

https://www.youtube.com/watch?v=v7fKfDp6cQs&ab_channel=PeladoNerd

How to learn operating systems?

1. Understanding high levels concepts and using a real operating system.

2. Implementing an operating system.

3. Using an instructional operating system.

https://www.studytonight.com/tests/?subject=operating-system

FAQ

Why do we need operating systems?

Even though the operating system produces an overhead, it is necessary because it reduces development time in user applications and that is beneficial to final users.

What is the difference between GNU/Linux and Unix?

Although Dennis Ritchie considers Linux as the de facto Unix system (Interview with Dennis M. Ritchie Manuel Benet, LinuxFocus, July 1999) and the current Unix commercial versions and Linux are POSIX complain, they don’t have the same codebase, they extended that specification in a different way, and of course, they have a different license.

What do I call Operating System API?

Some interfaces are in libraries (for instance, stdio.h or unistd.h) and for others, you have to use exec stuff. So, you have got to search for your problem.

Worked example in Linux.

As user view (https://man7.org/linux/man-pages/man1/kill.1.html)

kill -KILL 3333

From programmer's view (https://man7.org/linux/man-pages/man2/kill.2.html)

#include <signal.h>

int main() {
    kill(3333, SIGKILL);
    return 0;
}

Some references are https://www.kernel.org/doc/man-pages/ and ‘The Linux Programming Interface’ (published in October 2010, No Starch Press, ISBN 978-1-59327-220-3).

Operating system development and its architecture. Design Considerations.

Common operating system components

• PlantUML

  @startuml
  !theme mars
  User ..-> [Applications]: use
  node "Applications" {
    node "User interfaces" {
      [batch] 
      [command line]
      [GUI]
    }
  }
  note left of batch: For instance, job schedulers. Cron is job scheduler in UNIX-like.
  
  node "Operating system" {
    portin "System Calls" as SystemCalls
    [command line] -down-> SystemCalls
    [batch] -down-> SystemCalls
    [GUI] -down-> SystemCalls
    SystemCalls -down- [Process Management]
    SystemCalls -down- [Storage Management]
    SystemCalls -down- [Communication]
  SystemCalls -down- [Protection and security]
  SystemCalls -down- [Resource allocation]
  SystemCalls -down- [Input/Output Management]
  SystemCalls -down- [Accounting]
  SystemCalls -down- [Error detection]
  }
  @enduml

Operating system services

System calls are operations that the operating system makes for you.

Design Philosophy

Simple is beautiful sometimes called small-is-beautiful philosophy. Examples: Unix, Latin, RISC.

UNIX is done with only six basic system calls which are fork, executing another process, open, close, read, and write files.

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/unistd.h.html

Start off on complexity. Examples: Windows, Chinese, CISC.

Windows is done with every imaginable system call.

Operating system architectures. Operating system structures. Kernel models.

Monolithic systems

Layered systems

Microkernels

Client-Server Model

Virtual Machines

Exokernels

Modular Kernel

Low-level tools

Large Programming Projects

Vim, Make, C, GNU, Clang, GNU, C++,

Rust?

Instruction set architecture (ISA)

Because operating systems use low-level tools you as operating systems really have concerns about your ISA, but x86 is the most common so we are going to work with it. x86-32 and x86-64.

If you are an application programmer (including a compiler programmer), operating systems make a lot of work for your specification or requirements but they limited all work that you can in a computer.

Bootstrap process

Your programs and the kernel run on RAM, but who does load the kernel? Computers must implement a bootstrap process to load the operating system. But your computer needs Firmware which is software that is in persistent memory —it lives after your computer shut down. LILO and GRUP are bootloaders.

Bootstrapping (or booting) is the process to pass the control into software instead of raw hardware. The bootloader is the software that follows the CPU vendor protocol to be able to use machine resources and it is loaded by the firmware in the ROM (UEFI or BIOS). Developing that firmware is a vendor task, and developing the bootloader is an operating system’s developer task. The firmware is a de facto standard, so you must read the vendor manual. Remember, a lesson is “follow the standards”.

Modern computers use read-only memory (ROM) to load the kernel. EPROM and EEPROM are electric-erasable versions.

Every processor implements an instruction set (ISA), and the processor in their manuals specify what are the way to bootstrap a program —normally the kernel, but, you can run other programs. Today the most common ISA for general-purpose computers are IA32 and x86-64. For our purposes, Intel and AMD sets are identical.

For illustrations, we are focusing on general-purpose computers with IA32, but the general idea the following the protocol by specification is the same for other processor.

“Plug-and-play” is an auto-discovery mechanism where the hardware bus protocol must be updated to identify and initialize devices where the OS Kernel had to prove various IO ports. Plug-and-play buses are PCI family, USB, FireWire, PC Carc/PCMCIA.

BIOS or UEFI are special firmware that lives in the ROM and whose main mission is loading the bootloader.

BIOS is the original firmware for x86 computers, it provides some basic operations like screen output. The ultimate goal of OS is to get into protected mode.

Advanced Configuration and Power Interface (ACPI) is a standard that defines a platform-independent interface for hardware configuration. It ables you to plug-and-play and power management. As an OS kernel, ACPI consists of tables that contain platform configuration details that are accessible through a structure called the Root System Description Pointer. APIC details.

Unified Extensible Firmware Interface (UEFI) is a standard that has more capabilities than previous firmware. Unlike BIOS which works with Master Boot Records, UEFI works with GUID Partition Tables (GPT). Those features are a modular system, writing Unicode symbols, multi-multiboot services, and backward compatibility with BIOS. Some people see as drawbacks that the main driver is Microsoft and its “secure boot feature”. Vendors are migrating to UEFI, even though UEFI is large, complex, and has bugs.

You can check out a IA32 bootstrap with BIOS in ROM as follows

1. Someone or something starts the motherboard.

2. A circuit reset the IA32 processor.

3. IA32 processor starts executing instructions at 0xFFFFFFF0 immediately after a reset.

4. Computer manufacturers place a ROM in this address range to start the boot process. IA32 CPUs start off in ring 0 (kernel modes). It runs the BIOS.

5. BIOS runs some diagnostics on the hardware which is the Power-on Self Test, a.k.a. POST. It loads your bootloader in 0x7C00 address, you have a boot sector in only 512 bytes to load the next stage.

6. Bootloader. Here is when you go from the old archaic modes x86 or 8088 memory addressing modes into the real model. Disk partitions. Here you load the kernel. The bootloader can interact with the firmware but it is not in protected mode, which means you can interact with basic device drivers (keyboard and screen) but is not safe. When you pass to kernel, you cannot do anymore.

7. The kernel does some bootstrapping operations such as disk partitioning (MBR or GPT), memory virtual management settings, and others. Now kernel is responsible to interact with Device drivers and interrupts. Plug and Play come up by ACPI Standard.

8. The kernel starts the virtualization —Process management and memory management.

9. Happy user!

Firmware

Today, firmware can be BIOS or UEFI. If your firmware is BIOS, you have to write your bootloader, but you don’t on UEFI, because it has a bootloader manager called bcfg. UEFI has more benefits than a program that aims to load the kernel. Both you have got implement GDT, IDT, and so on.

Notes and references

interrupts

IRQ

Programmable Interrupt Controller (PIC)

Real Mode

Protected mode

https://wiki.archlinux.org/title/Arch_boot_process

https://www.youtube.com/watch?v=ejz7lNN5XG4&ab_channel=GunnarWolf

https://wiki.osdev.org/Interrupts

https://arjunsreedharan.org/post/99370248137/kernels-201-lets-write-a-kernel-with-keyboard

Mastering Embedded Linux Programming

https://wiki.osdev.org/BOOTBOOT

https://www.codeproject.com/Articles/664165/Writing-a-boot-loader-in-Assembly-and-C-Part

https://en.wikibooks.org/wiki/X86_Assembly/Bootloaders

https://wiki.osdev.org/Rolling_Your_Own_Bootloader

https://www.codejam.info/2021/08/super-fast-boot-linux-efistub.html

https://www.rodsbooks.com/efi-bootloaders/

https://wiki.osdev.org/UEFI

https://interrupt.memfault.com/blog/zero-to-main-1

https://fizuxchyk.wordpress.com/2024/06/13/nmbl-we-dont-need-a-bootloader/

https://developer.ibm.com/articles/l-linuxboot/

https://wiki.osdev.org/UEFI

https://stackoverflow.com/questions/58002542/how-does-uefi-boot-mode-boot-flows

"Beyond BIOS" book

https://github.com/torvalds/linux/blob/master/arch/x86/boot/main.c

Bootloader

EFISTUB

GRUB

Process management

Process description

A process is an instance of a program in execution.

A program from an operating system’s view is a bunch of isolated instructions on the disk.

Process data structures

Global Descriptor Table

ACPI MADT table

ELF

Machine state

Create

Destroy

Wait

Status

Process API

fork, exec, process tree, spawn

Direct Execution

Scheduling

The scheduling problem is hard.

CPU Scheduling

https://tontinton.com/posts/scheduling-internals/

Ostrich algorithm?

Goals

Concepts

Algorithms for batch processing

Arrival Time

Completion time

Burst time

Turn Around time

Waiting Time (W.T.).

First come, First served (FCFS)

Shortest-Job-Next

Priority Scheduling

Shortest Reaming Time

Expropiative version

Round Robin

Expropiative version

Quantum

Multiple-Level Queues Scheduling

Summarize

https://docs.google.com/spreadsheets/d/1SBEBDR7Lux3Xo-Jac3bKvISdmSSQdnNkYpb8zOeKKYI/edit?usp=sharing

CPU Scheduling – Jerry Breecher

Notes

https://www.tutorialspoint.com/operating_system/os_process_scheduling_algorithms.htm

http://perugini.cps.udayton.edu/teaching/courses/cps346/lecture_notes/scheduling.html

https://www.geeksforgeeks.org/program-fcfs-scheduling-set-1/

http://laurel.datsi.fi.upm.es/_media/docencia/asignaturas/dso/planificacion_2010_4pp.pdf

http://perugini.cps.udayton.edu/teaching/courses/cps346/lecture_notes/scheduling.html

http://www2.latech.edu/~box/os/ch05.pdf

https://cryptomex.org/SlidesSistOp/AdmonProcs.pdf

Segmentation. Stack-based memory allocation.

https://stackoverflow.com/questions/45303014/explanation-of-all-columns-in-size-command-output

Inter-process communication (IPC) models

It depends on what OSI model we are.

Message-passing

Example include , , .

Shared Memory or Shared-state

Example include and .

Note. Making a child i.e. fork() doesn’t mean sharing memory.

Inter-process communication (IPC) strategies

Interrupts & Signals

Interrupts, sometimes called traps, are

References

harismuneer. (2022, September 30). BelalHashmi-Assembly-Exercise-Solutions. Retrieved from https://github.com/harismuneer/BelalHashmi-Assembly-Exercise-Solutions

https://www.youtube.com/watch?v=54BrU82ANww&ab_channel=Computerphile

Microprocessor Design/Interrupts - Wikibooks, open books for an open world. (2022, September 16). Retrieved from https://en.wikibooks.org/wiki/Microprocessor_Design/Interrupts

x86 Assembly/Advanced Interrupts - Wikibooks, open books for an open world. (2022, September 16). Retrieved from https://en.wikibooks.org/wiki/X86_Assembly/Advanced_Interrupts

5.-IRQs and the PIT. (2017, January 15). Retrieved from http://www.jamesmolloy.co.uk/tutorial_html/5.-IRQs and the PIT.html

Interrupt Service Routine -Keyboard Interrupt Example | Hitch Hiker's Guide to Learning. (2022, September 30). Retrieved from https://www.hitchhikersguidetolearning.com/2021/05/09/interrupt-service-routine-keyboard-interrupt

Shared memory systems

Queue passing

Pipes

User mode

Example

ls | grep .c | wc -l

It means

stdin → ls → grep .c → wc -l → stdout

Programming mode

Named pipes

Worked example

1. Communicate two processes made in two different programming languages by pipes.

2. A starvation-free job-scheduling policy guarantees that no job waits indefinitely for service. Which job-scheduling policies is starvation-free? Why?

References

Johnny Huang » Linux IPC with Pipes. (2015, August 08). Retrieved from https://hzqtc.github.io/2012/07/linux-ipc-with-pipes.html

Sockets

“Socket” is a generalized interprocess communication that can exchange data between applications in a network.

Application Protocols are above sockets such as RPC (remote procedure calls), HTTP, FTP, REST, WebSocket …

Case study.

References and notes

https://en.wikipedia.org/wiki/Inter-process_communication

Worked examples

• Considerando la existencia de cuatro procesos (P1, P2, P3 y P4), un recurso R1 con dos ejemplares, un recurso R2 con tres ejemplares, y los recursos R3 y R4 cada uno con un ejemplar. Describa un escenario en el cual cada proceso tenga asignado al menos un recurso y esté solicitando al menos otro, de tal forma que genere un deadlock.

  Condiciones para generar un deadlock:

  1. Ciclo P1 → …. → P1.

  2. Cada proceso tenga asignado al menos un recurso y esté solicitando al menos otro. Los nodos adyacentes a un recurso son procesos y al revés.

  3. La cantidad de ejemplares es menor a la cantidad de aristas. Esto es la cantidad aristas es mayor a 7.

  graph LR
    P1((P1))
    P2((P2))
    P3((P3))
    P4((P4))
    R1[R1..]
    R2[R2...]
    R3[R3.]
    R4[R4.]
    P1-->R1
    R1-->P2
    P2-->R2
    R2-->P3
    P3-->R3
    R3-->P4
  	P4-->R4
    R4-->P1
    P3 --> R1
  

References

Chapter 7 Abraham Silberschatz Operating System Concepts 9th

Notes and references

https://www.cs.cmu.edu/~ralf/files.html

FAQ

Concurrency & Parallelism

“Concurrency” is a computing model (language concept) where a system executes $n$ computations during overlapping time periods at most $n$ ($< n$) processors and over on memory.

“Parallelism” is a computing model (hardware concept) where a system executes $n$ computations during overlapping time periods of at least $n$ ($\ge n$) processors and over on memory.

Multiprogramming refers to concurrency and parallelism, informally multiprogramming is a situation where you cannot say which event will happen first by looking at the program.

Time slicing.

Scheduler.

Examples of the systems that deal with multiprogramming are

• Distributed systems

• Operating systems

• Process

• Threads

Competitive and Independent concurrency is the event where two or more concurrent subsystems do not communicate with each other nor sync with each other neither. You don't worry about how to share the subsystem states and sync processes.

gantt
    title Competitive and Independent concurrency
    dateFormat m mm
    section Process are independent
    Process A           :a1, 0
    Process B           :a1, 0

Cooperative concurrency is the event where two or more concurrent subsystems communicate or sync with each other. You worry about and .

References

Reading 21: Concurrency. (2022, May 23). Retrieved from https://web.mit.edu/6.031/www/sp22/classes/21-concurrency

https://www.info.ucl.ac.be/~pvr/VanRoyChapter.pdf section 4.3 page 25

Classic problems

Reader & Writer

Race condition

A race condition or race hazard is the condition of a system where the system’s substantive behavior is dependent.

Non-determinism or Indeterminate.

Generally speaking, multiprogramming is a non-determinism situation.

Suppose two concurrent tasks:

print("yes")

print("no")

Because tasks run concurrently, the order of execution depends on the scheduler. So the output might be “yes no” o “no yes”.

Heisenbugs

Threads

While the processes are isolated, threads share the same data (). So, if you want multi lightweight tasks or need to share data easily with each other, a sound choice is to write a multithreaded process.

A thread is a flow of execution of a process. It said that a process is sequential when it only has a thread and if it has multiple execution flows is known as multi-threaded. The main thread is a special one because it is the entry and exit point of a process.

The threads are in the user-level, kernel-level threads, and lightweight threads. In the Linux case, a lightweight process and a kernel-level thread are the same but from a different stand of point, the former from the Kernel case, the latter from the user space.

Tasks.

Models.

"What are Linux Processes, Threads, Light Weight Processes, and Process State." 27 Feb. 2023, www.thegeekstuff.com/2013/11/linux-process-and-threads.

"Process - (Kernel) Thread (Lightweight processes - LWP)." Datacadamia - Data and Co, 5 Apr. 2020, datacadamia.com/os/process/thread.

Lightweight processes.

An introduction to programming with threads, Andrew Birrell, Research Report 35, Digital Equipment Corporation Systems Research Center, 1989.

Why not? The Problem with Threads, Edward A. Lee, UC Berkeley, 2006

Synchronization

In computer systems, synchronization is the relationship between events. We have seen problems if the events were free, so your system has to deal with two constraints:

Producer and consumer.

Serialization.

Mutual exclusion.

Spurious wakeup

Piece of advice for making safe Synchronization

• Confinement

• Immutability

• Use existing threadsafe data types
  • Synchronization

Dining Philosophers

Mechanisms for Synchronization

Mutual exclusion

Locks

Critical section

Thread Safety

Interrupts & Signals

Promises

Semaphores

Monitor Brinch Hansen (1973), Hoare (1974)

Monitors

Actor model

Dekker’s algorithm

Peterson's algorithm

Remove Method Invocation (RPC)

CORBA Web Services

J2EE

COM/DCOM

Websocket

Banker’s algorithm

References

The Little Book of Semaphores, Allen B. Downey.

FAQ

What is the difference between fork and thread?

Memory management

Background

Memory addressing

Heap

Dynamic Relocation

Static Relocation

MMU (memory management unit)

Protection

Sharing

Base

Worked examples

Continuous Memory Allocation

Paging

Structure of the Page Table

Swapping

Virtual Memory

ZRAM

Debugging

Valgrind Home. (2022, November 14). Retrieved from https://valgrind.org

References

• http://www.sc.ehu.es/acwlaroa/SO2/Apuntes/Cap4.pdf

• Memory Management

• https://www.inf.ed.ac.uk/teaching/courses/os/slides/09-memory16.pdf

• Administración de la memoria – C ejemplos

• http://www.dc.fi.udc.es/~so-grado/SO-Memoria.pdf

Memory Management

• https://www.dc.fi.udc.es/~so-grado/2018-19/Temas/SO-Memoria.pdf

• Block swap

• https://mht.technology/post/block-swap/

The persistence layer. Storage management.

Physical devices

Drivers

Network

Blocks

File system

Partitioning Table GPT/MBR

File systems.

FUSE

https://wiki.archlinux.org/title/Solid_state_drive

https://www.youtube.com/watch?v=P-9osuSdGGI&ab_channel=SpaceRex

Potpourri. (2022, October 27). Retrieved from https://missing.csail.mit.edu/2020/potpourri

https://en.wikipedia.org/wiki/List_of_file_systems

Input/Output device management

Text rendering

VGA text mode

Interruptions and traps

Interrupt descriptor table

Keyboard & Mouse

Graphics

No.

Windows managers and compositors

Assignment

References

[Intel (2011)] Intel 64 and IA-32 Architectures Software Developer’s Manual, Com- bined Volumes: 1, 2A, 2B, 3A and 3B. Intel Corporation (2011).

Virtualization

Virtualization is a mechanism that provides access transparency to the operating system, networks, storage, processes or physical machines.

Process Sandbox

As was disccused in Process managment, OS provides a degree of virtualization which disable some calls to hardware and direct modifications to other proceses.

OS

Fargate, Lambda, or Fly Machines

Firecracker microVM or a V8

Running Untrusted Code

Functions as service, Playgrounds, and multi tenants are applications

gVisor.

WASI sandboxing

Browsers

https://blog.cloudflare.com/sandboxing-in-linux-with-zero-lines-of-code

untrusted source

Linux containers

FREBSD Jails

SELinux

https://healeycodes.com/running-untrusted-python-code

https://github.com/cloudflare/sandbox

seccomp

Firejail

systrace

AppArmor

https://healeycodes.com/running-untrusted-python-code

https://github.com/openedx/codejail

https://www.youtube.com/watch?v=YWZPpWVUyTc

https://dev.to/narasimha1997/building-a-secure-sandboxed-environment-for-executing-untrusted-code-7e8

https://www.opensourceforu.com/2016/07/many-approaches-sandboxing-linux/

http://web.archive.org/web/20141122112516/http://blog.technologyofcontent.com/2011/04/sandboxing-for-multi-tenant-applications

https://doc.pypy.org/en/latest/sandbox.html

https://github.com/openedx/codejail

Emulators

An emulator mimics an instruction set architecture at behavior over a host architecture.

DosBox

https://www.isardvdi.com/

API Emulator

An API emulator mimics the behavior of one operating system on another. Therefore, these emulators implement the methods of the guest system based on the host kernel. For example, WINE re-implements the Win32 API, and WSL has re-implemented POSIX.

Virtual machines

Architecture

Hypervisors

A hypervisor is a software component that manages virtual machines. There are two main types: bare-metal hypervisors (also known as type 1) and software-based hypervisors (also known as type 2).

Virtual Box

VMware

QEMU

libvirt

KVM

Clients

Vagrant

Cookbook with QEMU, libvirt and KVM

Storage virtualization

Network virtualization

Data virtualization

Application virtualization

Application virtualization is a technology that allows you to run software applications in isolated environments, separate from the underlying operating system and other applications on a computer.

Package managers

NixOS

Bundle managers

Flatpack

Snap

X2Go

• Cameyo

• Ceedo

• Citrix XenApp

• Microsoft App-V

• Numecent Cloudpaging

• Oracle Secure Global Desktop

• Sandboxie

• Turbo (software) (formerly Spoon and Xenocode)

• Symantec Workspace Virtualization

• VMware ThinApp

• V2 Cloud

Containers

Docker

LXC

Podman

AppJail in BSD

Docker,OpenVZ/Virtuozzo,and LXC/LXD.

Kubernetes and Docker Swarm

WSL

VNC, RDP and ssh -X

NoVNC

https://www.nomachine.com/

Wayland Westorn

Apache Guacamole

NoVNC

https://github.com/sanchezcarlosjr/desktop-app-to-webapp

Rootless

https://github.com/fsquillace/junest

X

References

Mastering KVM Virtualization by Vedran Dakic, Humble Devassy Chirammal, Prasad Mukhedkar, Anil Vettathu

https://www.docker.com/blog/app-virtualization-docker/

https://aws.amazon.com/what-is/virtualization/

https://www.youtube.com/watch?v=WhENXFkiOlI

Linux

Bresnahan, C., & Blum, R. (2019). CompTIA Linux+ Study Guide:
Exam XK0-004. Sybex, a Wiley brand.
Bresnahan, C., & Blum, R. (2020). Lpic-1 - Linux Professional
Institute Certification Study Guide: Exam 101-500 and exam
102-500. Sybex.
Brunson, R., & Walberg, S. (2016). CompTIA Linux+ / Lpic-1 Cert
Guide. Pearson Certification. [Clásica]
Inshanally, P. (2018). CompTIA linux+ certification guide. Packt
Publishing.
Jordan, T., & Strohmayer, S. (2020). CompTIA linux+ certification
all-in-one exam guide: Exam XK0-004. McGraw-Hill.

Serna, M. y Allende, S. (2020). Sistemas operativos: Linux.
Jorge Sarmiento Editor - Universitas. Recuperado de:

https://elibro.net/es/ereader/uabc/175148?page=1

Linux as user

Unix philosophy

https://blog.kellybrazil.com/2019/11/26/bringing-the-unix-philosophy-to-the-21st-century/

Shell scripting

Morris, J., McCubbin, C., & Page, R. (2018). Hands-on Data Science with command line: Automate Everyday Data Science tasks using command line tools. PACKT Publishing Limited

Janssens, J., & O'Reilly, T. (2021). Data Science at the command line: Obtain, scrub, explore, and model data with Unix Power Tools. Recuperado de: https://datascienceatthecommandline.com/2e/index.html
Pelz, O. (2018). Fundamentals of linux: Explore the essentials
of the Linux Command Line. Packt Publishing.

Basics | Cheat sheets | Linux Command Library. (2023, January 07). Retrieved from https://linuxcommandlibrary.com

Command-line Environment. (2022, October 27). Retrieved from https://missing.csail.mit.edu/2020/command-line

pv

dialog

textual

rich

dialog --title "Input Box" --inputbox "Input" 0 0 2>output.txt

Dotfiles

dotly

Syscalls

Wang, K. C. (2018). Systems programming in Unix/linux.
Recuperado de:
https://link.springer.com/book/10.1007%2F978-3-319-
92429-8#about

strace

Process

systemd

Kernel Modules

You’re going to write a kernel module in Linux, so you must be careful since an error could crash your system. You can build a safe environment with docker.

docker run --volume "$(pwd):/temp" -it --rm archlinux /bin/bash

You can list all kernel modules using

lsmod

If you are interested in kernel modules, read [1]. Essentially you need to install

pacman -S gcc kmod linux-headers make

You can hello world file in

References

[1] The Linux Kernel Module Programming Guide. (2022, December 25). Retrieved from https://sysprog21.github.io/lkmpg/#introduction

Suspend a process and resume it later

How can I resume a stopped job in Linux? (2022, December 31). Retrieved from https://superuser.com/questions/268230/how-can-i-resume-a-stopped-job-in-linux

sk. (2022). How To Suspend A Process And Resume It Later In Linux - OSTechNix. OSTechNix. Retrieved from https://ostechnix.com/suspend-process-resume-later-linux

Daemons

Systemd

Parallel process

prog1 & prog2 &

https://stackoverflow.com/questions/3004811/how-do-you-run-multiple-programs-in-parallel-from-a-bash-script

Background process

# ctrl+z
bg # last program to background

make &

nohup

nohup --help

User commands

Interlude

https://missing.csail.mit.edu/2020/potpourri/

IPC

Interlude

Pipes

Interlude

Queue Messaging

ipcrm -q

Interlude

Shared Memory

Interlude

Sockets in POSIX

Socket system call

socket(domain, type, protocol) is a system call that creates a new socket where domain={AF_UNIX, AF_INET, AF_INET6, …}, type={SOCK_STREAM, SOCK_DATAGRAM, …}, listen=0.

Some Socket Domains. It is also known as the “Address families”.

You can find a full list of address families on the address_families(7) - Linux manual page. (2022, September 26). Retrieved from https://man7.org/linux/man-pages/man7/address_families.7.html

Socket Types. You can find it as “Communication styles” too.

If type=SOCK_STREAM, then the sequence is the following:

sequenceDiagram
		    Note over Passive socket (server), Active socket (client): Create sockets
        Passive socket (server) ->> Passive socket (server): socket()
        Passive socket (server) ->> Passive socket (server): bind()
        Passive socket (server) ->> Passive socket (server): listen()
        loop block until client connects
          Passive socket (server) ->> Passive socket (server): accept()         
        end
		    Note over Passive socket (server), Active socket (client): Start connection
        Active socket (client)->> Active socket (client): socket()
        Active socket (client) ->> Passive socket (server): connect()
		    Note over Passive socket (server), Active socket (client): Start communication
        loop they transfers data
          Passive socket (server) --> Active socket (client): write()/read() or send()/recv()
        end
		    Note over Passive socket (server), Active socket (client): Close communication
        Passive socket (server) ->> Passive socket (server): close()
        Active socket (client)->> Active socket (client): close()

Reuse Port

How do I use setsockopt(SO_REUSEADDR)? (2022, September 26). Retrieved from https://stackoverflow.com/questions/24194961/how-do-i-use-setsockoptso-reuseaddr

Interlude

🕸️Tic-tac-toe Web on POSIX Sockets and WebRTC

References

Sockets (The GNU C Library). (2022, September 26). Retrieved from https://www.gnu.org/software/libc/manual/html_node/Sockets.html

TCP/IP Sockets in C: Practical Guide for Programmers, Second Edition

Sockets (The GNU C Library). (2022, September 24). Retrieved from https://www.gnu.org/software/libc/manual/html_node/Sockets.html

Beej's Guide to Network Programming, Using Internet Sockets

Coverage: [OSCJ8] Chapter 5 (pp. 193-239); [OSC8] Chapter 5 (pp. 183-223)?

rpc(3) - Linux manual page. (2022, September 26). Retrieved from https://man7.org/linux/man-pages/man3/rpc.3.html

Semaphores

Writers and readers

File system

Summary

References and notes

Process manager

FAQ

What is the difference between POSIX, Single UNIX Specification, and Open Group Base Specifications?

https://unix.stackexchange.com/questions/14368/difference-between-posix-single-unix-specification-and-open-group-base-specifi

Windows 11

We introduce you to the most common APIs. If you want to know more, please consult the documentation.

Android

Interacting with Firmware

Live USBs

BIOS

UEFI

Operating systems for specific tasks

Cloud

Security

Synchronization tools

Write your modern simple operating system from scratch, but it won’t be easy.