Ideas for FreeLdr

[admin]

valerius

Joined: 03 Apr 2005
Posts: 50
Location: Elizovo, Kamchatka, Russia
Posted: Wed May 09, 2007 8:08 am

For FreeLdr, it would be desirable to have the following features:

1) To have possibility to see debug messages from MicroFSD and FreeLdr. If we will output these messages
to screen, then, when the system boots, the messages appear wery quickly and are quickly erased by kernel'
and servers' messages. It is also posiible to output these messages to comm port, but nowadays, there are
many computers without a comm port. So, an idea suggests itself, to have a feature like "dmesg" command
for Linux kernel, or the one like "copy devibms506$ con" in OS/2. In other words, we could make a log buffer,
which exists over all the osFree boot process, and which can be examined later.

For collecting all messages from different places, I suggest to have a log server, which maintains a log buffer
and collects messages from other servers. This log server must to be included in Personality Neutral Services.
Also, I propose some mechanism for collecting messages from loader and microfsd's. I propose a log buffer
to be created by boot partition's MicroFSD. The messages are output to the buffer by microfsd itself. A microfsd
includes a special log() function which outputs messages to the buffer. When a FreeLdr is started by microfsd,
a pointer to the log buffer is passed to the loader -- in some register, or right after FileTable structure (to be
defined). And, along with the buffer, a pointer to the log() function is passed. When the multiboot kernel starts,
the buffer is passed to multiboot kernel. I propose to pass it as ordinary multiboot module in mods[] list. When
L4 kernel is started, it starts sigma0 and root servers. The root server starts log server. Then log server acquires
the log buffer and continues logging messages to it. The question remains, how to collect messages from kickstart
and L4 kernel, and is this needed really?

2) I want to have the possibility for the loader to load ordinary OS/2 kernel (os2krnl). For that, there can be two
possibilities: a) the possibility to exchange multiboot kernel support to custom kernel (like os2krnl) support. This
variant is to divide the loader into two parts: general and custom. The custom part is different for multiboot
kernels and custom kernels, like OS/2 one. The general part is common. b) and different variant -- to base the
loader solely on Multiboot standard. I.e., the loader loads a multiboot kernel, which can be a Linux kernel or
kickstart for L4 microkernel. For custom kinds of kernels, like OS/2 kernel, a special multiboot kernel can be
created. It relocates itself to the address within 1st megabyte (conventional memory), switches back to the real
mode and starts IBM's os2ldr, which, in turn, starts OS/2 kernel. For this, the OS/2 kernel specific multiboot
kernel must have FileTable and BPB structures passed from freeldr. For this, I propose to pass these structures,
along with log buffer, in mods[] array, in the end of it.

The problem for multiboot kernel is that some kernels, like Linux one, doesn't understand these additional
modules (FileTable, BPB and log buffer). The question is: how to make the kernels, which are not aware of
them, to be not confused by these structures? -- I propose to add these structures to the very end of mods[]
list. Linux kernel understands only two modules -- the Linux kernel itself and the Initrd (am I right?). And
kickstart understands only 3 first modules (the L4 kernel, sigma0 server and the root server). The next modules
are ignored by kickstart, they are only passed in the Kernel Interface Page by bootinfo structure. Then, the
root server and other servers (like log server) can obtain other modules from the bootinfo structure.

Any comments, suggestions?
_________________
WBR,
Valery V. Sedletski

[admin]

valerius

Joined: 03 Apr 2005
Posts: 50
Location: Elizovo, Kamchatka, Russia
Posted: Wed May 09, 2007 11:24 am

The standard interface between MicroFSD and Loader is:

1) Flags in DH:

Code: Select all

DH
boot mode flags:

bit 0 (NOVOLIO) on indicates that the mini-FSD does not use MFSH_DOVOLIO.
bit 1 (RIPL) on indicates that boot volume is not local (RIPL boot)
bit 2 (MINIFSD) on indicates that a mini-FSD is present.
bit 3 (RESERVED)
bit 4 (MICROFSD) on indicates that a micro-FSD is present.
bits 5-7 are reserved and MUST be zero.

2)

Code: Select all

DL

drive number for the boot disk. This parameter is ignored if either the NOVOLIO or MINIFSD bits are zero.

3) Pointer to boot media BPB in DS:SI
4) Pointer to FileTable structure in ES:DI.

The FileTable structure is the following:

Code: Select all

struct FileTable {
    unsigned short ft_cfiles; /* # of entries in this table            */
    unsigned short ft_ldrseg; /* paragraph # where OS2LDR is loaded    */
    unsigned long  ft_ldrlen; /* length of OS2LDR in bytes              */
    unsigned short ft_museg;  /* paragraph # where microFSD is loaded  */
    unsigned long  ft_mulen;  /* length of microFSD in bytes            */
    unsigned short ft_mfsseg; /* paragraph # where miniFSD is loaded    */
    unsigned long  ft_mfslen; /* length of miniFSD in bytes            */
    unsigned short ft_ripseg; /* paragraph # where RIPL data is loaded  */
    unsigned long  ft_riplen; /* length of RIPL data in bytes          */

    /* The next four elements are pointers to microFSD entry points    */

    unsigned short (far *ft_muOpen)
                    (char far *pName, unsigned long far *pulFileSize);
    unsigned long (far *ft_muRead)
                    (long loffseek, char far *pBuf, unsigned long cbBuf);
    unsigned long (far *ft_muClose)(void);
    unsigned long (far *ft_muTerminate)(void);
};

How we can add an extension to this interface? (It can be necessary, if MicroFSD loads additional
files, besides microfsd, minifsd, loader and RIPL data, or exports additional functions, like log()
function). I propose the following. The ft_cfiles parameter can be equal to 3 (no RIPL data), or 4
(there is a RIPL data). If it is =4, but ft_ripseg=0 and ft_ripseg=0, then RIPL data is not valid. And
how to add extensions:

Code: Select all

struct FileTable --> struct FileTableExt
Code:
struct FileTableExt {
  struct FileTable  fte_filetab;
    ...<extensions>
    ...<extensions>
} fte;

If fte.fte_filetab.ft_cfiles > 4, (RIPL data params can be == 0 in this case, or not), then the
"<extensions>" (additional parameters) are valid.

This way, we can add additional files/functions to the filetable.

Additionally, we can have "new type" microfsds, which support extensions, and "legacy" ones, that doesn't.
"Legacy" microfsd's have ft_cfiles parameter == 3 or 4, the "new" ones have them greater than 4. And then,
from "<extensions>" parameters FreeLdr can obtain addresses of files in memory and functions.

If microfsd gives an address of log buffer and log() fuhnction to the loader, then the loader uses them, if it
doesn't, then the loader creates buffer by itself and uses it's own log() function.

Suggestions? Is these extensions really needed? What do you think?
_________________
WBR,
Valery V. Sedletski

[admin]

prokushev

Site Admin

Joined: 25 Nov 2003
Posts: 100
Posted: Sun May 13, 2007 5:28 am

Thread can be continued in Log Server forum.

[admin]

prokushev

Site Admin

Joined: 25 Nov 2003
Posts: 100
Posted: Sun May 13, 2007 6:49 am

For FreeLdr, it would be desirable to have the following features:

2) I want to have the possibility for the loader to load ordinary OS/2 kernel (os2krnl). For that, there can
be two possibilities: a) the possibility to exchange multiboot kernel support to custom kernel (like os2krnl)
support. This variant is to divide the loader into two parts: general and custom. The custom part is different
for multiboot kernels and custom kernels, like OS/2 one. The general part is common. b) and different variant
-- to base the loader solely on Multiboot standard. I.e., the loader loads a multiboot kernel, which can be a
Linux kernel or kickstart for L4 microkernel. For custom kinds of kernels, like OS/2 kernel, a special multiboot
kernel can be created. It relocates itself to the address within 1st megabyte (conventional memory), switches
back to the real mode and starts IBM's os2ldr, which, in turn, starts OS/2 kernel. For this, the OS/2 kernel
specific multiboot kernel must have FileTable and BPB structures passed from freeldr. For this, I propose to
pass these structures, along with log buffer, in mods[] array, in the end of it.

The problem for multiboot kernel is that some kernels, like Linux one, doesn't understand these additional
modules (FileTable, BPB and log buffer). The question is: how to make the kernels, which are not aware of
them, to be not confused by these structures? -- I propose to add these structures to the very end of mods[]
list. Linux kernel understands only two modules -- the Linux kernel itself and the Initrd (am I right?). And
kickstart understands only 3 first modules (the L4 kernel, sigma0 server and the root server). The next modules
are ignored by kickstart, they are only passed in the Kernel Interface Page by bootinfo structure. Then, the
root server and other servers (like log server) can obtain other modules from the bootinfo structure.

Any comments, suggestions?

Well. I prefer to have only multiboot part in loader. Custom kernels format better to handle using something
like kickstart for L4. According data passing. I don't see any problems with passing data as module. We can
just add fake module to end of module list.

[admin]

valerius

Joined: 03 Apr 2005
Posts: 50
Location: Elizovo, Kamchatka, Russia
Posted: Sun May 13, 2007 9:32 am

Yes, and I prefer to make os2krnl support as a multiboot kernel. It' s more appropriate
_________________
WBR,
Valery V. Sedletski

[admin]

valerius

Joined: 03 Apr 2005
Posts: 50
Location: Elizovo, Kamchatka, Russia
Posted: Sun May 13, 2007 9:43 am

I propose to use FileTable structure as a means of linkage -- different modules, like loader stage0,
stage1, microfsd and microxfd's place here the addresses of its functions, and other modules can
take them from there. (For details, read on)

We need:
1) Loader to be in a multi-segment EXE format (like DOS EXE), to be extensible. Now it consists
only from one segment.
2) Different modules to be possible to call functions from other modules
3) Our loader to be backward compatible with IBM's microFSD's and our microFSD's to be backward
compatible with a single-segment loader, like IBM's. I.e., our loader must be sucessfully loaded by
IBM's microfsd, and our microfsd must load a single-segment loader.
4) A MicroFSD must contain only a minimal set of filesystem-specific functions.

So, I propose the following.

1) Divide a microfsd to the so-called loader stage0 and microfsd itself. Microfsd must contain
standard mu_Open(), mu_Read(), mu_Close(), mu_Terminate(), and also Initialization routine
mu_Init(), mount routine mu_Mount(), and probablu, other functions. The stage0 is called so
because it is not 1-st stage of loader, and by half is a part of a microfsd. Stage0 consists of
general functions, like mu_Start(), mu_Log(), functions for reading sectors from disk, printk(),
video functions etc. Stage0 and microfsd are in different files, but they are loaded by the
bootsector by one map -- stage0 sectors firstly, and microfsd sectors secondly -- in a contoguous
run of sectors (in memory).

First, an assembler startup of stage0 starts mu_Start() stage0 function. The bootsector is relocated
from the address 0x7c00 to before the stage0 beginning. mu_Start() sets the BPB location, fills FileTable
with stage0 functions addresses and then calls mu_Init() functions from the microfsd. It's entry point must
be at the very microfsd beginning. A pointer to a FileTable and Flags (to be then passed in DH register)
are passed to mu_Init() through function parameters. mu_Init() then sets flags for DH register and FileTable
by microfsd mu_*() functions. After that, stage0 and microfsd know each other entry points.

Then stage0 parses its small config file which tells it which files to which addresses to load and what is
the role of each file (minifsd, loader, or something else), and loader stage1 entry point to call.

After that, stage0 loads a number of files (minifsd, loader stage1, and, probably, more), and calls stage1,
passing it standard parameters (BPB, FileTable, flags in DH register and drive number in DL register).

Then, stage1 starts. It accept parameters. The role of stage1 is to incorporate microxfd functions, and load
stage2 by them. Stage1 is a raw binary executable, and stage2 can have an arbitrary format, supported by
microxfd's.

Microxfd stands for Micro eXecutable Format Driver (Yuri, I propose to use such a term). It is a simple, raw
binary format executable. It consists of only one segment, and its size cant exceed 64kb. Microxfd must contain
three functions -- Init one, Load one, and Recognize one. The Init function plays the role analogous to mu_Init()
microfsd function. It's purpose to set parameters and to do linkage to other loader functions. Recognize function
must return 0 if a microxfd recognized EXE format as its own, and 1 otherwise. Load function must load actual
executable and return entry point and stack pointer.

Stage1 must load stage2. For this, it uses microxfd's. A list of all microxfd's is in the config file, muxfds.lst.
Stage1 tries to use all muxfd's in order of their precedence in the list. First, muxfd is loaded, then Init() function
is called from it. After that, the muxfd functions are available to the loader, and vice versa. Then Recognize()
function is called. If it returns 1, then the next muxfd is loaded, and so on. If a format is recognized, then it is
loaded.

After loading stage2, stage1 passes FileTable, BPB, DH, DL, pushing these parameters to stage2 stack. Then,
assembler startup of stage2 takes them from here, and loader is initialized.

So, the main idea is -- to use a FileTable as an interface table for location of all files and functions in loader,
not only microfsd.

I propose to place the pointers to additional 'Files' and 'Functions' structures at the end of FileTable structure.
The "Files" list can be like in mods[] list in multiboot structure.
_________________
WBR,
Valery V. Sedletski

[admin]

prokushev

Site Admin

Joined: 25 Nov 2003
Posts: 100
Posted: Mon May 14, 2007 12:16 pm

Agree in general.