CEST OS. Your modern simple operating system from scratch on the Web.

Tags

Overview

📍

Our goal is to make a modern simple kernel for learning purposes, not a fully optimized state-of-art operating system such as Ubuntu, Arch Linux, etc.

Demo

https://cest.sanchezcarlosjr.com/

Code

https://github.com/sanchezcarlosjr/cest

Keywords

operating system, kernel

Introduction

Background

Goals

Domain understanding

Functional requirements

A shell with input from an ASCII keyboard and output to the screen.

Non-Functional Requirements

Arch x86-64 bits.

English.

Our tech stack is NASM, clang, Make, C, and C++.

Since we follow standards, we deploy on the UEFI. Our operating system will be incompatible with BIOS.

We use QEMU for testing purposes because it is easier for kernel development than other options such as VirtualBox.

We follow POSIX.

The Open Group Base Specifications Issue 7, 2018 edition. (2022, September 11). Retrieved from https://pubs.opengroup.org/onlinepubs/9699919799.2018edition

Design

Microkernel structure.

A user.

General-purpose operating system.

No networking.

x86-64 arquitechture.

cest
│
└───boot # uefi boot
   startup.h
└───lib # libraries such that users can build on top of them
   └───libc # standard C library
      stdio.h
      stdlib.h
      string.h
└───kernel # load arquitechture specific interfaces
  └───interrupt
  └───memory
  └───proc

Ecosystem

toolchain

x86-64

Linker Script

Boot process

The boot process could be better, but there are no standards. We describe to you the boot process for OS developers in x86-64. The boot process is highly dependent on the platform. If today works, maybe tomorrow does not.

// Boot program. It is a firmware whom works is load the kernel.
// You should check out its protocol in order to achieve it.
EFI. BIOS. 

// Microkernel tasks. Bootstrapping.
bootstrap::allocator(stivale2_struct);
bootstrap::translator(stivale2_struct);
bootstrap::enable_virtualaddr();
bootstrap::heap(0x10);
bootstrap::boot_arch(); // gpt
bootstrap::interrupts();
bootstrap::enable_interrupts();

// Procceses
// ...

// IO Drivers
bootstrap::screen(stivale2_struct);
bootstrap::keyboard();
bootstrap::acpi(stivale2_struct);
bootstrap::pci();
bootstrap::rtl8139();

jmolloy. (2022, December 25). JMTK. Retrieved from https://github.com/jmolloy/JMTK/blob/master/src/x86/gdt.c

GDTR and Segmentation in x86_64. (2022, December 25). Retrieved from https://stackoverflow.com/questions/60801674/gdtr-and-segmentation-in-x86-64

Setting up a GDT - intermezzOS. (2019, October 06). Retrieved from https://intermezzos.github.io/book/first-edition/setting-up-a-gdt.html

winksaville. (2022, December 25). baremetal-x86_64. Retrieved from https://github.com/winksaville/baremetal-x86_64/blob/master/gdt.c

x86-64

Notes on x86-64 Assembly and Machine Code. (2022, December 25). Retrieved from https://gist.github.com/lancejpollard/1db84c233bcd849b237df76b3a6c4d9e

Lessons

Step 0 Install tools

Clone repository with submodules

git clone --recurse-submodules https://github.com/sanchezcarlosjr/cest.git

When you’ve installed docker, run

docker compose up

Checking process.

Debugging

QEMU Options

In-built

GDB

GDB usage — QEMU documentation. (2022, December 21). Retrieved from https://qemu-project.gitlab.io/qemu/system/gdb.html

QEMU/Debugging with QEMU - Wikibooks, open books for an open world. (2022, December 11). Retrieved from https://en.wikibooks.org/wiki/QEMU/Debugging_with_QEMU

https://wiki.osdev.org/Qemu

socat -,echo=0,icanon=0 unix-connect:qemu-monitor-socket

Connect to running qemu instance with qemu monitor. (2022, December 25). Retrieved from https://unix.stackexchange.com/questions/426652/connect-to-running-qemu-instance-with-qemu-monitor

Assignments

A lot of tutorials speak a lot about the bootloader, so we are going to ignore the bootloader since we use the “bootboot” protocol. We don’t worry about the legacy mode (real modes, protected modes, …). In the “boot boot” protocol, we are in the “Long mode”

CS =0008 0000000000000000 0000ffff 00209800 DPL=0 CS64 [---]

Printing strings and numbers

Virtualizing processes by hardware

Intel provides mechanisms to manage processes, those mechanisms are GDT, LDT, and TSS —segmentation, and task switching. Since we are developing in Intel x86 (x86-32 and x86-64), we must enable them. In other words, we’re going to fill up tables.

Interruptions

Virtualizing processes by software

Memory

Stack

Heap

Scheduling

References

https://rua.ua.es/dspace/bitstream/10045/124698/1/Desarrollo_de_un_kernel_academico_para_arquitecturas_Martinez_Garcia_Ernesto.pdf

ecomaikgolf. (2022, October 29). alma. Retrieved from https://github.com/ecomaikgolf/alma

https://forum.osdev.org/viewtopic.php?f=1&t=32412

Absurdponcho. (2022, October 29). PonchoOS. Retrieved from https://github.com/Absurdponcho/PonchoOS/tree/main

[edk2] Handling hardware interrupts in EFI. (2022, October 29). Retrieved from https://edk2-devel.narkive.com/ht2Bda8v/edk2-handling-hardware-interrupts-in-efi

JSLinux. (2021, January 09). Retrieved from https://bellard.org/jslinux

AT&T assembly

. (2022, December 24). ThatOSDev Archive. Youtube. Retrieved from https://www.youtube.com/@thatosdevarchive2087

Thatosdev. (2022, December 24). UEFI-Tuts. Retrieved from https://github.com/ThatOSDev/UEFI-Tuts

bzt / bootboot · GitLab. (2022, December 25). Retrieved from https://gitlab.com/bztsrc/bootboot

OpenSecurityTraining2. (2022, July 14). Class Intro - Architecture 2001: x86-64 OS Internals. Youtube. Retrieved from https://www.youtube.com/playlist?list=PLUFkSN0XLZ-myVyCmMvfz_W5Z5SauI3cN

levex. (2022, December 25). osdev. Retrieved from https://github.com/levex/osdev