This week I went on the CppCast podcast to talk about SerenityOS and building an operating system from scratch in C++. It was my first time on a podcast so I was a bit nervous about it, but I think it went okay. :)
SerentiyOS has struck me as a neat project, although I haven’t dug into it very much. Implementing a PC operating system today that even attempts to be usable in the modern computing environment is definitely cool, and hopefully will even prove useful. Particularly, implementing a novel browser, even a bare-bones one, seems like a really important project in this day when the web has become such an important computing platform and every browser but Firefox is using Google’s engine. I didn’t realize the creator had struggled with alcohol addiction, and it’s good to hear that they are keeping sober.
I’m mildly surprised to learn (from the downvoted comment https://lobste.rs/s/rkx214/serenityos_second_year#c_w7eurv ) that SerentiyOS has a 32 bit kernel. That does seem like a design decision that arbitrarily limits what can be done with the operating system - is it even easier to write a 32-bit than a 64-bit kernel, on modern hardware with modern tools? Does a 32-bit kernel necessarily prevent userspace processes from accessing more than 4GB of memory?
The way I see it, 64-bit support is just another feature. There are countless features I want to implement. The system is not hitting any of the limitations imposed by the 32-bit kernel, so it’s not a priority to work on it right now. That said, the userspace is 64-bit clean outside of architecture specific things. Large parts of it are compiled and tested on Linux by our CI.
We’ll make a port to x86_64 when we start hitting those limitations, or when someone feels like working on it anyway. The SerenityOS project is 100% driven by fun and curiosity, something not everyone can relate to :)
This seems worthy of comment, then, in a world where we are worried about the number of independent browser engines. Could LibWeb or whatever the browser engine is called be extracted for use elsewhere, or is it tied to SerenityOS in some fashion?
A big difference between this and other small browser projects is that Andreas worked on webkit and safari for many many years, so has a more complete knowledge of browsers than most other small projects.
It could certainly be extracted with some effort. It’s implemented on top of various SerenityOS libraries (LibCore, LibGfx, LibGUI, LibIPC, LibProtocol) so most of the work would probably be building a platform abstraction layer where these things would be pluggable at build-time.
a platform abstraction layer where these things would be pluggable at build-time
That could feasibly be (subsets of) all those Lib* bundles reimplemented on top of POSIX/Win32/what-have-you.
I think Andreas Kling is an inspiration when it comes to having a methodology where he never gets stuck in details, and focuses on one result at a time.
It has definitely helped me keep going with a project of such large scope. Also helpful is the time pressure from recording screencasts. I’m actually amazed at how much I’ve improved as a developer from regularly recording myself.
Hey thanks Glaeqen! I’m so proud of all the contributors and the great work they’ve been putting in. And of course, immediately after posting the video I thought of many more things to show.. :)
is it true that always_inline always causes a function to be inlined? my impression is that it’s up to the compiler still, and in some/many cases things marked always_inline aren’t inlined, but i haven’t actually investigated this at all…
I’m so glad to hear that. The first and second time I heard about pledge/unveil I didn’t “get it”, so if I’m finally able to explain it clearly I think that’s good! :)
Purgeable memory is a fantastic concept, and it would be great to see more software make use of it.
While I think this would benefit greatly from being implemented in the kernel rather than as a userspace hack, it’s always awesome to see userspace hacks, so kudos to Chris for this implementation! :)
I’m currently using Qemu for osdev, and I was wondering what system you were using to print to your linux terminal. Could make some debugging easier for me down the line ^^
Hi zge! If you run qemu with the -debugcon stdio command-line option, you can output to I/O port 0xe9 from your OS and it will show up in your Linux terminal :)
This is so awesome! I’m so glad you kept copies of these. I’ve tried hunting down my old teenage home pages from the ’90s but with very little success.
I also love how personal each design iteration of your website feels. Thank you for sharing this :)
A bunch of people have asked me to do a walk-through of the Serenity OS boot process, and this is it. It covers everything from the initial assembly code entry (where we receive control from the boot loader) up to the point where we launch the user-space GUI.
Most of it is stepping through the C++ kernel boot sequence and discussing what each part is for, and the various things that get initialized along the way.
If you’re not familiar, Serenity is a new operating system that I’ve been building since October 2018 that combines an old-school Unix feel with an old-school 90s office software look, all written in modern C++ :)
When a project I am working on starts losing steam, I find that adding new visualizations can give me the motivation that I need to keep working on it.
Strongly agree with this.
Also, cool project. It’s always interesting to see all the different levels of NIH that people consider “from scratch” :)
I realized that I’m gonna have to start building an IDE if I ever want to switch to using my own operating system full time. Since I spend 80% of my computer time programming, I don’t want to be stuck using a crappy single-file text editor. So I’ve started building an IDE. :)
> writes operating system from scratch with no limits or forced compatibility in terms of design and functionality
> makes it look like Windows 95
Jokes aside, @awesomekling, great work on this! Keep it up and be brave to try out new ideas! Don’t be stuck in the “old” models, even though they might be dominating the GUI on all major platforms.
Hey FRIGN! I’m definitely trying out some new ideas, like the /proc filesystem using JSON to make exchanging data between kernel and userspace really pleasant. One of the other developers, Sergey Bugaev, has introduced system support for Markdown which we now use for the built-in man pages. Just two random things I quite like about the system.
As far as the GUI goes though, I really like the style so far, although I wish my icon drawing skills would improve faster :)
How long did you work on the project before making the first commit? As I understand it you had EXT2 filesystem support and some other things before you committed?
They were in separate repos at first, so I can tell you!
I spent 7 days writing the initial ext2 implementation, and 2 days writing an ELF loader.
I’m less sure about the initial kernel, as I wrote that with very little structure while going through a rough period, but a couple of weeks.
All of my free time in the last year, basically. I was unemployed for the first 6 months of the project, so I was working full time on this, 7 days a week. After that I got a job which slowed me down a bit. I still like to imagine that I’m keeping decent pace though. :)
pretty incredible, congrads!
in my most productive coding years, I am pushing 80-90K lines of code (business apps).
But you seem to be doing quite a bit more, and more complexity!.
What’s your technical background, and what if any ‘philosophy/principles’ do you have for the OS?
Hey vladislavp! My technical background is being a self-taught high school dropout who learned C++ from open source development (KDE) and then worked at Nokia (Qt project) and Apple (WebKit). Nowadays I work at a small research lab in Sweden and do this project with my free time. :)
Damn! That is an insane amount of work! Did the author of this work on it like every single day? I guess some people play video games and read books, and this person wrote a whole operating system. Really more than just an OS, we’re talking kernel, frame buffer, a really neat GUI toolkit (that was written parallel on Linux first), porting some other tools into it, support for multi processes and process monitoring. Very cool.
Hi djsumdog! If you check my GitHub history you’ll find that yes, I work on this basically every single day :)
I do miss playing video games, but I’m really proud of how far the system has come in one year. It’s been worth it!
This week I went on the CppCast podcast to talk about SerenityOS and building an operating system from scratch in C++. It was my first time on a podcast so I was a bit nervous about it, but I think it went okay. :)
SerentiyOS has struck me as a neat project, although I haven’t dug into it very much. Implementing a PC operating system today that even attempts to be usable in the modern computing environment is definitely cool, and hopefully will even prove useful. Particularly, implementing a novel browser, even a bare-bones one, seems like a really important project in this day when the web has become such an important computing platform and every browser but Firefox is using Google’s engine. I didn’t realize the creator had struggled with alcohol addiction, and it’s good to hear that they are keeping sober.
I’m mildly surprised to learn (from the downvoted comment https://lobste.rs/s/rkx214/serenityos_second_year#c_w7eurv ) that SerentiyOS has a 32 bit kernel. That does seem like a design decision that arbitrarily limits what can be done with the operating system - is it even easier to write a 32-bit than a 64-bit kernel, on modern hardware with modern tools? Does a 32-bit kernel necessarily prevent userspace processes from accessing more than 4GB of memory?
The way I see it, 64-bit support is just another feature. There are countless features I want to implement. The system is not hitting any of the limitations imposed by the 32-bit kernel, so it’s not a priority to work on it right now. That said, the userspace is 64-bit clean outside of architecture specific things. Large parts of it are compiled and tested on Linux by our CI.
We’ll make a port to x86_64 when we start hitting those limitations, or when someone feels like working on it anyway. The SerenityOS project is 100% driven by fun and curiosity, something not everyone can relate to :)
I’m interested in this too, which is why the downvoted comment missed an opportunity to raise that question in a respectful way.
Edit this comment provides some background
https://lobste.rs/s/rkx214/serenityos_second_year#c_jzljgc
What browser engine does the web browser use? Did they write their own?
They’re writing their own. In the youtube channel you can see the vlogs of the implementation, it is very inspiring and fun.
This seems worthy of comment, then, in a world where we are worried about the number of independent browser engines. Could LibWeb or whatever the browser engine is called be extracted for use elsewhere, or is it tied to SerenityOS in some fashion?
Maybe, but it’s no more complete or compliant than many other small browser projects.
A big difference between this and other small browser projects is that Andreas worked on webkit and safari for many many years, so has a more complete knowledge of browsers than most other small projects.
It could certainly be extracted with some effort. It’s implemented on top of various SerenityOS libraries (LibCore, LibGfx, LibGUI, LibIPC, LibProtocol) so most of the work would probably be building a platform abstraction layer where these things would be pluggable at build-time.
I think Andreas Kling is an inspiration when it comes to having a methodology where he never gets stuck in details, and focuses on one result at a time.
Wow, that’s nice to hear! :^)
It has definitely helped me keep going with a project of such large scope. Also helpful is the time pressure from recording screencasts. I’m actually amazed at how much I’ve improved as a developer from regularly recording myself.
As always, excellent work. Great job Andreas (and, obviously, all contributors)!
Hey thanks Glaeqen! I’m so proud of all the contributors and the great work they’ve been putting in. And of course, immediately after posting the video I thought of many more things to show.. :)
is it true that always_inline always causes a function to be inlined? my impression is that it’s up to the compiler still, and in some/many cases things marked always_inline aren’t inlined, but i haven’t actually investigated this at all…
There are various things that will prevent the compiler from inlining even in the presence of “always_inline”. For example:
You can look at the LLVM source code to see what causes it to disable inlining. :)
Working on the JavaScript engine for my operating system.
I finally found a way to learn how JavaScript actually works. Turns out all I had to do was start implementing it from scratch :)
How good is the “with” keyword!
This is quite neat!
I was watching the youtube channel couple days ago and was wondering exactly about how to develop apps for it. Thanks a lot for sharing.
Anyone has a tutorial for running it without actually building it? Like a release for virtual machines or something.
Hi soapdog! You’ll have to build it yourself. The whole process is mostly scripted and usually takes less than 30 minutes to finish the first time. :)
Build instructions can be found here: https://github.com/SerenityOS/serenity/blob/master/Documentation/BuildInstructions.md
This post gets an upvote for its clear explanation on how pledge and unveil works.
I’m so glad to hear that. The first and second time I heard about pledge/unveil I didn’t “get it”, so if I’m finally able to explain it clearly I think that’s good! :)
Purgeable memory is a fantastic concept, and it would be great to see more software make use of it.
While I think this would benefit greatly from being implemented in the kernel rather than as a userspace hack, it’s always awesome to see userspace hacks, so kudos to Chris for this implementation! :)
Also, sidenote: the name “purgeable” (or actually “purgable” [sic]) comes from macOS. Windows has the same feature under the name Offer/Reclaim: https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-offervirtualmemory
I’m currently using Qemu for osdev, and I was wondering what system you were using to print to your linux terminal. Could make some debugging easier for me down the line ^^
Hi zge! If you run qemu with the
-debugcon stdiocommand-line option, you can output to I/O port0xe9from your OS and it will show up in your Linux terminal :)This is so awesome! I’m so glad you kept copies of these. I’ve tried hunting down my old teenage home pages from the ’90s but with very little success.
I also love how personal each design iteration of your website feels. Thank you for sharing this :)
Hello friends!
A bunch of people have asked me to do a walk-through of the Serenity OS boot process, and this is it. It covers everything from the initial assembly code entry (where we receive control from the boot loader) up to the point where we launch the user-space GUI.
Most of it is stepping through the C++ kernel boot sequence and discussing what each part is for, and the various things that get initialized along the way.
If you’re not familiar, Serenity is a new operating system that I’ve been building since October 2018 that combines an old-school Unix feel with an old-school 90s office software look, all written in modern C++ :)
Strongly agree with this.
Also, cool project. It’s always interesting to see all the different levels of NIH that people consider “from scratch” :)
I realized that I’m gonna have to start building an IDE if I ever want to switch to using my own operating system full time. Since I spend 80% of my computer time programming, I don’t want to be stuck using a crappy single-file text editor. So I’ve started building an IDE. :)
> writes operating system from scratch with no limits or forced compatibility in terms of design and functionality > makes it look like Windows 95
Jokes aside, @awesomekling, great work on this! Keep it up and be brave to try out new ideas! Don’t be stuck in the “old” models, even though they might be dominating the GUI on all major platforms.
Hey FRIGN! I’m definitely trying out some new ideas, like the /proc filesystem using JSON to make exchanging data between kernel and userspace really pleasant. One of the other developers, Sergey Bugaev, has introduced system support for Markdown which we now use for the built-in man pages. Just two random things I quite like about the system.
As far as the GUI goes though, I really like the style so far, although I wish my icon drawing skills would improve faster :)
These hand sign for SIGSTOP and SIGCONT are great though. :)
Hello friends! (And thanks jcs for the invite.)
I’m the author of this OS and I’m happy to answer any questions you might have about it :)
If you don’t have any questions that’s okay too, I’ll just say thanks for checking out my project!
How long did you work on the project before making the first commit? As I understand it you had EXT2 filesystem support and some other things before you committed?
They were in separate repos at first, so I can tell you! I spent 7 days writing the initial ext2 implementation, and 2 days writing an ELF loader. I’m less sure about the initial kernel, as I wrote that with very little structure while going through a rough period, but a couple of weeks.
Very cool stuff. How many man hours went into this?
All of my free time in the last year, basically. I was unemployed for the first 6 months of the project, so I was working full time on this, 7 days a week. After that I got a job which slowed me down a bit. I still like to imagine that I’m keeping decent pace though. :)
Amazing! I’m curious, how long did you work on it before importing it into git, and are you at all concerned about burnout at this pace?
pretty incredible, congrads! in my most productive coding years, I am pushing 80-90K lines of code (business apps). But you seem to be doing quite a bit more, and more complexity!.
What’s your technical background, and what if any ‘philosophy/principles’ do you have for the OS?
Hey vladislavp! My technical background is being a self-taught high school dropout who learned C++ from open source development (KDE) and then worked at Nokia (Qt project) and Apple (WebKit). Nowadays I work at a small research lab in Sweden and do this project with my free time. :)
Long version, if someone is interested: https://www.youtube.com/watch?v=ncTesyJsDvU
Damn! That is an insane amount of work! Did the author of this work on it like every single day? I guess some people play video games and read books, and this person wrote a whole operating system. Really more than just an OS, we’re talking kernel, frame buffer, a really neat GUI toolkit (that was written parallel on Linux first), porting some other tools into it, support for multi processes and process monitoring. Very cool.
Hi djsumdog! If you check my GitHub history you’ll find that yes, I work on this basically every single day :) I do miss playing video games, but I’m really proud of how far the system has come in one year. It’s been worth it!
[wrong comment]