8086 style Segmentation + Paging in the same system!

Thread Starter

AnalogDigitalDesigner

Joined Jan 22, 2018
121
(Please delete the other thread not this one). I prefer to keep it in gen elec.

Hi there friends,

I am an electronics engineer and I am designing and building a 16bit CPU and Minicomputer from scratch using 74 series logic.

I have a question about the implementation of virtual memory. Basically right now the CPU has memory segmentation. The virtual address is Segment:Offset where the segment is then shifted left by 8 bits and the offset added to that to form the physical addr. So I have a 24bit physical space.

However I also want to support paging. My question is, is it OK to add paging on top of this segmented model? What I mean is, the programmer would see the Segment:Offset virtual memory, and the CPU would take the 24 bit address formed by that and compute a paged address from it.

Please note the important fact that my segmentation is 8086 style - non protected. It's just an arithmetic operation of shifting the segment register and adding the offset. It has no GDT or LDT.

Is it normal to do this? I think the pentium does this but not sure.

So would it be ok to have segment + paging in the same system?

Please help.
 

nsaspook

Joined Aug 27, 2009
7,992
(Please delete the other thread not this one). I prefer to keep it in gen elec.

Hi there friends,

I am an electronics engineer and I am designing and building a 16bit CPU and Minicomputer from scratch using 74 series logic.

I have a question about the implementation of virtual memory. Basically right now the CPU has memory segmentation. The virtual address is Segment:Offset where the segment is then shifted left by 8 bits and the offset added to that to form the physical addr. So I have a 24bit physical space.

However I also want to support paging. My question is, is it OK to add paging on top of this segmented model? What I mean is, the programmer would see the Segment:Offset virtual memory, and the CPU would take the 24 bit address formed by that and compute a paged address from it.

Please note the important fact that my segmentation is 8086 style - non protected. It's just an arithmetic operation of shifting the segment register and adding the offset. It has no GDT or LDT.

Is it normal to do this? I think the pentium does this but not sure.

So would it be ok to have segment + paging in the same system?

Please help.
Sure, you can page on top of segmentation or below it.
http://nob.cs.ucdavis.edu/classes/ecs150-2008-02/handouts/memory/mm-segpag.html

That was pretty common before most 32-bit OS models eliminated segments with modern x86 processors.
https://pdfs.semanticscholar.org/presentation/d238/5927e9194a3450a40f2ef56bbce515d9b5e4.pdf
http://lass.cs.umass.edu/~shenoy/courses/fall10/lectures/Lec15_notes.pdf
 
Last edited:

Thread Starter

AnalogDigitalDesigner

Joined Jan 22, 2018
121
Hey NSASpook (hehehe). Thanks for that. Let me ask you to kindly answer this: My segmentation is 8086 like, that is, it offers no GDT or segment limits for protection. It's just a plain segment: offset addition. Has any architecture ever implemented this kind of simple segmentation together with paging? Or is this just a figment of my imagination? Also, will this be alright to program with or will it be a monster?

The programmer will see the segment: offset registers and each segment is independent up to 256 bytes. The MMU then transforms this into a 24 bit virtual address. Paging then takes this and you know what happens then. Finally a 24 bit address is formed.

Is this worthwhile or should I just remove segmentation and just do paging with 16 bit virtual space per process ?
 

nsaspook

Joined Aug 27, 2009
7,992
Hey NSASpook (hehehe). Thanks for that. Let me ask you to kindly answer this: My segmentation is 8086 like, that is, it offers no GDT or segment limits for protection. It's just a plain segment: offset addition. Has any architecture ever implemented this kind of simple segmentation together with paging? Or is this just a figment of my imagination? Also, will this be alright to program with or will it be a monster?

The programmer will see the segment: offset registers and each segment is independent up to 256 bytes. The MMU then transforms this into a 24 bit virtual address. Paging then takes this and you know what happens then. Finally a 24 bit address is formed.

Is this worthwhile or should I just remove segmentation and just do paging with 16 bit virtual space per process ?
I would think most systems would have paging physical memory first with segments on top of that.

It's easier to build the hardware and to write a simple OS with segmentation because it provides a natural organization of computer resources per process, stack, etc ... Memory size is not a bottleneck today so I rarely see systems paging to physical disk unless they are seriously memory over-committed. For paging you need some type of system to fault or interrupt the processor when 'something' isn't there. It helps to also have a virtual machine capability for re-running an instruction that caused one of those faults after the page is in memory.
 

Thread Starter

AnalogDigitalDesigner

Joined Jan 22, 2018
121
I would think most systems would have paging physical memory first with segments on top of that.

It's easier to build the hardware and to write a simple OS with segmentation because it provides a natural organization of computer resources per process, stack, etc ... Memory size is not a bottleneck today so I rarely see systems paging to physical disk unless they are seriously memory over-committed. For paging you need some type of system to fault or interrupt the processor when 'something' isn't there. It helps to also have a virtual machine capability for re-running an instruction that caused one of those faults after the page is in memory.

Do you think it's worthwhile and/or a good thing to do, to have 8086 segmentation with paging? What I am trying to say is that my segmentation by itself offers no protection because it does not have a limit check. It's just a S:O pair and that's it. I did this to save on backplane lines. For protection then I need paging on top of that (after it). My question is, should I just throw away the segmentation and just do 16bit paging, or should I leave the segmentation and do paging after it? What do you think?
 

nsaspook

Joined Aug 27, 2009
7,992
Do you think it's worthwhile and/or a good thing to do, to have 8086 segmentation with paging? What I am trying to say is that my segmentation by itself offers no protection because it does not have a limit check. It's just a S:O pair and that's it. I did this to save on backplane lines. For protection then I need paging on top of that (after it). My question is, should I just throw away the segmentation and just do 16bit paging, or should I leave the segmentation and do paging after it? What do you think?

Paging just by itself doesn't provide protection (paging permits a programs memory to be noncontinuous in physical memory), it's a capability that can be easily added to the system requirements. You will need some sort of hardware assisted 'fence' (bounds) registers, a resident monitor with hardware special privileged instruction capability with trap or interrupts, protected page tables and some sort of look-aside or associative cache of you want a reasonable speed. From the user program view of memory, segments on top of paging (Segmented Paging) is logical if paging is required. The logical address space is a collection of segments in a program.

Paged segmentation (leave the segmentation and do paging after it) helps to solve the wasted, fragmented memory problem during allocation but that comes with a complexity cost.

What do I think? That you should read up on the subject so you can decide.

A good school book on the subject.


 

Thread Starter

AnalogDigitalDesigner

Joined Jan 22, 2018
121
Paging just by itself doesn't provide protection (paging permits a programs memory to be noncontinuous in physical memory), it's a capability that can be easily added to the system requirements. You will need some sort of hardware assisted 'fence' (bounds) registers, a resident monitor with hardware special privileged instruction capability with trap or interrupts, protected page tables and some sort of look-aside or associative cache of you want a reasonable speed. From the user program view of memory, segments on top of paging (Segmented Paging) is logical if paging is required. The logical address space is a collection of segments in a program.

Paged segmentation (leave the segmentation and do paging after it) helps to solve the wasted, fragmented memory problem during allocation but that comes with a complexity cost.

What do I think? That you should read up on the subject so you can decide.

A good school book on the subject.



It's not true that paging offers no memory protection. Paging's purpose in to offer protection as well. If you consider the virtual address, it can be implemented so that the kernel has a chunk of the address at all times, and the user processes can use the other chunk, so that each process has the entire chunk for itself. If he user-process tries to access the kernel a fault happens.

Another way is to have each process see the entire virtual address space, so that no process can touch another process's memory since it's outside the virtual space, unless you map some pages to the same physical space. The kernel then also has the entire virtual space when it's loaded.

the second way offers total memory protection between processes as long as no 2 processes are mapped to the same physical address.

I will have a look at that book although I have already read many books on this subject.

Thank you!
 

nsaspook

Joined Aug 27, 2009
7,992
It's not true that paging offers no memory protection. Paging's purpose in to offer protection as well. If you consider the virtual address, it can be implemented so that the kernel has a chunk of the address at all times, and the user processes can use the other chunk, so that each process has the entire chunk for itself. If he user-process tries to access the kernel a fault happens.

Another way is to have each process see the entire virtual address space, so that no process can touch another process's memory since it's outside the virtual space, unless you map some pages to the same physical space. The kernel then also has the entire virtual space when it's loaded.

the second way offers total memory protection between processes as long as no 2 processes are mapped to the same physical address.

I will have a look at that book although I have already read many books on this subject.

Thank you!
In a computer hardware/OS system the segmentation, paging and VM are different subsystems that can work together to provide memory protection. As you say it 'can' be implemented but there is no requirement for just paging. Demand Paging is a common concept used to implement virtual memory where you don't require the entire process (all pages) to be in memory before the program execution begins. Virtual memory is just a way of presenting physical memory so that each process has a separate memory space. This usually requires an MMU (Address translation hardware) to enforce hardware protection but an MMU is not needed for simple paging and virtual memory can exist without paging.
 

KL7AJ

Joined Nov 4, 2008
2,229
(Please delete the other thread not this one). I prefer to keep it in gen elec.

Hi there friends,

I am an electronics engineer and I am designing and building a 16bit CPU and Minicomputer from scratch using 74 series logic.

I have a question about the implementation of virtual memory. Basically right now the CPU has memory segmentation. The virtual address is Segment:Offset where the segment is then shifted left by 8 bits and the offset added to that to form the physical addr. So I have a 24bit physical space.

However I also want to support paging. My question is, is it OK to add paging on top of this segmented model? What I mean is, the programmer would see the Segment:Offset virtual memory, and the CPU would take the 24 bit address formed by that and compute a paged address from it.

Please note the important fact that my segmentation is 8086 style - non protected. It's just an arithmetic operation of shifting the segment register and adding the offset. It has no GDT or LDT.

Is it normal to do this? I think the pentium does this but not sure.

So would it be ok to have segment + paging in the same system?

Please help.
I admire your dedication and tenacity!
 

Thread Starter

AnalogDigitalDesigner

Joined Jan 22, 2018
121
In a computer hardware/OS system the segmentation, paging and VM are different subsystems that can work together to provide memory protection. As you say it 'can' be implemented but there is no requirement for just paging. Demand Paging is a common concept used to implement virtual memory where you don't require the entire process (all pages) to be in memory before the program execution begins. Virtual memory is just a way of presenting physical memory so that each process has a separate memory space. This usually requires an MMU (Address translation hardware) to enforce hardware protection but an MMU is not needed for simple paging and virtual memory can exist without paging.

I do understand what both segmentation and (demand)paging are..... :p In fact I have already implemented demand paging in hardware using a page table base register, and a fast RAM chip located inside the CPU where the pages are searched and physical addresses + faults are fetched/raised.

I also know that they can be used together but this is usually done with proper segmentation tables that provide both the base and the segment limit, which I don't have.That's why I said my segmentation is 8086 like: it doesn't have limit checking.

I guess I will just create two modes, one for pure 8086-like segmentation and another for pure demand paging.

The 8086 segmentation is just a hack, it was done to save on pins rather than to provide any protection, so it's really worthless. I did it because I wanted to run 8086 programs on my minicomputer, but it would be weird to force this onto all programs.

if I do segment+paging, each program would see 4 segments only, CS, DS, SS, ES, and various offsets combined with that. Segments and offsets are 16 bits each and would form a 24 bit virtual address. This virtual address is fed into the page table. Lets say the offset into the page is 18 bits, then the other 6 bits is combined with 6 bits from the page table base register and fed into the page table (fast ram located inside the cpu). The page table then outputs a 16 bit address, 4 bits of which are used for page faults, so that 12 bits become the high part of the physical address, which are combined with the low 12 bits from the virtual address to form the final 24 bit physical address.

This is the idea. I am not sure yet if this is a good idea though. I think it would be trickier for the kernel to load pages if it sees the segment model, even though it can control the page table register and use either its, or the user-mode process' value.
 

Thread Starter

AnalogDigitalDesigner

Joined Jan 22, 2018
121
I admire your dedication and tenacity!

I am very driven when I am doing a project that I know will last for as long as I will, and I want to do it properly. I will connect this system to the internet and allow guests to login and play with it. It's a major project of my own and it's done for my own pleasure :)
 

nsaspook

Joined Aug 27, 2009
7,992
I do understand what both segmentation and (demand)paging are..... :p In fact I have already implemented demand paging in hardware using a page table base register, and a fast RAM chip located inside the CPU where the pages are searched and physical addresses + faults are fetched/raised.

I also know that they can be used together but this is usually done with proper segmentation tables that provide both the base and the segment limit, which I don't have.That's why I said my segmentation is 8086 like: it doesn't have limit checking.

I guess I will just create two modes, one for pure 8086-like segmentation and another for pure demand paging.

The 8086 segmentation is just a hack, it was done to save on pins rather than to provide any protection, so it's really worthless. I did it because I wanted to run 8086 programs on my minicomputer, but it would be weird to force this onto all programs.

if I do segment+paging, each program would see 4 segments only, CS, DS, SS, ES, and various offsets combined with that. Segments and offsets are 16 bits each and would form a 24 bit virtual address. This virtual address is fed into the page table. Lets say the offset into the page is 18 bits, then the other 6 bits is combined with 6 bits from the page table base register and fed into the page table (fast ram located inside the cpu). The page table then outputs a 16 bit address, 4 bits of which are used for page faults, so that 12 bits become the high part of the physical address, which are combined with the low 12 bits from the virtual address to form the final 24 bit physical address.

This is the idea. I am not sure yet if this is a good idea though. I think it would be trickier for the kernel to load pages if it sees the segment model, even though it can control the page table register and use either its, or the user-mode process' value.
Now that we've clarified the facts after finally hearing hardware details that really matter in understanding the true condition of hardware it's easier to see why you asked.

8086 segmentation was always a hack from the first fab'd chip so I wouldn't worry about it being good/bad.
http://www.os2museum.com/wp/will-the-real-real-mode-please-stand-up/
main-qimg-281cce575ae2f27ffef7272705316cd2.jpeg
The question is can your segment+paging be done efficiently without proper hardware segmentation tables with paging.
https://cs.nyu.edu/courses/spring09/V22.0202-002/lectures/lecture-22.html

It will be trickier and slower because the needed segment table emulation (for paging support) replacement algorithms with low fault rates will be more complex than page replacement algorithms like LRU. If you have the hardware (stack of page numbers in a linked list or at least a page reference bit) for efficient LRU then the paged 8086 mode could at least improve on its inefficiency vs pure 8086-like segmentation.

Modern OSes "do not use" true segmentation so making 'pure' modes is a good choice if you don't have paging AND full segmentation hardware like the Pentium.
 
Last edited:

Thread Starter

AnalogDigitalDesigner

Joined Jan 22, 2018
121
Now that we've clarified the facts after finally hearing hardware details that really matter in understanding the true condition of hardware it's easier to see why you asked.

8086 segmentation was always a hack from the first fab'd chip so I wouldn't worry about it being good/bad.
http://www.os2museum.com/wp/will-the-real-real-mode-please-stand-up/
View attachment 160610
The question is can your segment+paging be done efficiently without proper hardware segmentation tables with paging.
https://cs.nyu.edu/courses/spring09/V22.0202-002/lectures/lecture-22.html

It will be trickier and slower because the needed segment table emulation (for paging support) replacement algorithms with low fault rates will be more complex than page replacement algorithms like LRU. If you have the hardware (stack of page numbers in a linked list or at least a page reference bit) for efficient LRU then the paged 8086 mode could at least improve on its inefficiency vs pure 8086-like segmentation.

Modern OSes "do not use" true segmentation so making 'pure' modes is a good choice if you don't have paging AND full segmentation hardware like the Pentium.

I am not sure you understand what I am saying, maybe you are not reading my posts :)

I do have hardware for paging. Like I said I have cache memory inside the CPU and a page table register that separates tables per process, and these are used in making sure that paging is fast. As fast as possible.

On the other hand, for segmentation, since I don't have limit checking, it is as fast as possible, because it's just a segment register and an offset which are shifted and added together.

What I really want to know is whether combining these two makes any sense at all. Do you understand it?

The CPU sees the segment :Offset pair, and so when the kernel needs to load a process lets say, it would have a table in memory of what page belong to each process. But it only can access this memory by using the S:O pair of addresses because this come before paging. So Lets say it uses the data segment to store this. So it has a segment: Offset pair which locates the table that lists each process/page pair, and based on that it loads the pages, but the pages would have to be also addresses as S: O pairs, because thats all the OS sees. So for a programmer, they see only S: O pairs rather than a linear virtual address.

I am not sure how useful this really is, and I think it must be rather unpleasant.

I've already built a whole 30x20cm wirewrap board worth of registers including segmentation registers and I don't want to throw it all away. Maybe the best solution is to have 2 modes, one for pure segmentation, and another for pure paging. Paging would have a 16bit virtual address.

Have a look at this monster:


See what I am trying to say? If you're still up for it, bring it on! :)
 
Last edited:

nsaspook

Joined Aug 27, 2009
7,992
I am not sure you understand what I am saying, maybe you are not reading my posts :)

I do have hardware for paging. Like I said I have cache memory inside the CPU and a page table register that separates tables per process, and these are used in making sure that paging is fast. As fast as possible.

On the other hand, for segmentation, since I don't have limit checking, it is as fast as possible, because it's just a segment register and an offset which are shifted and added together.

What I really want to know is whether combining these two makes any sense at all. Do you understand it?

The CPU sees the segment :Offset pair, and so when the kernel needs to load a process lets say, it would have a table in memory of what page belong to each process. But it only can access this memory by using the S:O pair of addresses because this come before paging. So Lets say it uses the data segment to store this. So it has a segment: Offset pair which locates the table that lists each process/page pair, and based on that it loads the pages, but the pages would have to be also addresses as S: O pairs, because thats all the OS sees. So for a programmer, they see only S: O pairs rather than a linear virtual address.

I am not sure how useful this really is, and I think it must be rather unpleasant.

I've already built a whole 30x20cm wirewrap board worth of registers including segmentation registers and I don't want to throw it all away. Maybe the best solution is to have 2 modes, one for pure segmentation, and another for pure paging. Paging would have a 16bit virtual address.

Have a look at this monster:


See what I am trying to say? If you're still up for it, bring it on! :)

I've already stated that 'pure' modes is a good choice given your hardware and tried to show the complexity of combining the two within the limitations of that hardware. As for how useful and unpleasant we can look at the past and present to see that Operating Systems that depended on hardware segmentation when memory was expensive and/or limited ability to address memory are few like OpenBSD or dead like OS/2 Warp instead of using demand paging for simplicity and portability like modern Linux. 8086 segmentation is a hack for limitations of the architecture. You need it to run 8086 programs but what's done with segmentation can be better done with paging, plus it gives a whole lot more. Your lack of segment limit checking means it can't solve the "protect a process from itself" problem so it adds no real added capability to the mix beyond what paging can do like solving the "protect processes from each other" problem, the "position independent code" problem, the "physical address space fragmentation" problem, etc ...

It's your project so only you can decide what's useful.
 

Thread Starter

AnalogDigitalDesigner

Joined Jan 22, 2018
121
I've already stated that 'pure' modes is a good choice given your hardware and tried to show the complexity of combining the two within the limitations of that hardware. As for how useful and unpleasant we can look at the past and present to see that Operating Systems that depended on hardware segmentation when memory was expensive and/or limited ability to address memory are few like OpenBSD or dead like OS/2 Warp instead of using demand paging for simplicity and portability like modern Linux. 8086 segmentation is a hack for limitations of the architecture. You need it to run 8086 programs but what's done with segmentation can be better done with paging, plus it gives a whole lot more. Your lack of segment limit checking means it can't solve the "protect a process from itself" problem so it adds no real added capability to the mix beyond what paging can do like solving the "protect processes from each other" problem, the "position independent code" problem, the "physical address space fragmentation" problem, etc ...

It's your project so only you can decide what's useful.
Fo sho
 
Top