Ghost Kernel

klange lately posted a video about his project ToaruOS. Great work and inspiration!

*Wohoo!* The update is done. Took quite some time: it introduces changes in the kernel, a new scheduler, and also a rewrite of the window server. These had to be revised a lot - yet there are still some bugs (which are already being hounded). :-)

The new scheduler implementation is based on wait and run queues. The internal system call handler interface was refurbished and the handling cleaned up. Changing the scheduler required modifying some of the handlers, as they were implemented based on the assumption that the "current thread" given by the tasking facilities is consistent; with the new queue implementation, a task that is told to wait is immediately moved to the wait queue, thus being no more the _current thread_. A system call handler now gets the last executed thread passed as a parameter and can use it immediately (without asking the schedulers for the current thread). This will also help improve performance of some interprocess functions and made it possible to improve the idling of the scheduler.

The window server was completely rewritten and cleansed from the obsolete interface, and a new (variable-sized-messages) based interface was introduced. This simplifies communication a lot. The old windowmanager application is now obsolete.

Next things to do will be fixing a bug that freezes the window server for some reason and then extending the UI interface to make it possible to implement the first applications that actually have a working user interface.

I did some major changes to the website, including a new comment section (and better markdown functionality). You can now comment on news posts and see what other users say about a specific topic. Also, there are now links accessible for each single news post.

*About the documentation*: I'm working on a dynamic part that generates documentation from the markup files in the project repository. Once this is finished, the documentation part will be back.

Lately I had some issues where the PS2 driver process froze for unrecognizable reasons. And it didn't freeze all the time - only when you did a quick series of command inputs. This led to a nice and long debugging time: the process did not hang in userspace, so it was obviously a problem in the kernel. There weren't many changes in the kernel API before, except... well, the new variable messaging implementation.

When a variably sized message is sent, the kernel looks for inside a map that contains the message queue head for the target process where the message will be inserted. When a process dies, it had inserted NULL into that map. And well, the next time accessing the map, it accessed a null pointer and things went weird.

Whats the lesson? Use assert or add checks everywhere you feel sure that this pointer points to a valid value now. You definitely don't want this kind of bugs. But if you have them, you might like my solution.

Monitoring application

So how did I find the cause of this. Well, when a process blocks, there is an instance of a so-called "waiter" attached to its structure. This waiter tries to do a specific operation (in this case, sending a message) each time the process would be scheduled. There are no kernel threads in Ghost, so they are all compressed to operations as small as possible that happen between two timeslices. The bug was in the routine that handled the sending, so the waiter would wait forever to send the message.

To get a monitoring application to see what process does what (kind of task manager you know from Windows, but outside of the kernel). This application (written in plain Java) connects to the TCP socket that QEMU can expose and that is internally linked to the COM1 port of the virtual machine. Like this, I can simply read bytes that are written from the kernel to COM1, and read bytes that I send there via the socket. I then defined a small protocol, that allows the kernel to send events. These events for example happen when a task is put to sleep (a waiter is appended). The task status is then updated and you can see it in the task area of the monitoring tool.

The tool also shows what filesystem nodes are cached in the kernel, what process uses how much CPU, and some other nice things. When the tool is a little more advanced (and the code erfined), I might make it more generic, I think other developers could profit of it too.

A little rudimentary by now, but you should get the idea. :-) This view for example shows the list of running threads, their binary source and their CPU usage (id 0 is the idle thread):

• Idle task uses "HLT" to prevent 100% processor usage
• fixed some problems in the standard I/O implementation, piping now works
• Directory reading functionality is reworked
• Spawner and others uses variable messages
• Color in the terminal, and a fancy start screen
• Minor bugfixes here and there
• g_atomic_wait is now g_atomic_lock
• g_spawn has slightly changed, its now possible to specify in and out descriptors of a process when spawning it