A Medley of Potpourri

Tuesday, June 24, 2025

Architecture of Windows NT

From Wikipedia, the free encyclopedia

The architecture of Windows NT, a line of operating systems produced and sold by Microsoft, is a layered design that consists of two main components, user mode and kernel mode. It is a preemptive, reentrant multitasking operating system, which has been designed to work with uniprocessor and symmetrical multiprocessor (SMP)-based computers. To process input/output (I/O) requests, it uses packet-driven I/O, which utilizes I/O request packets (IRPs) and asynchronous I/O. Starting with Windows XP, Microsoft began making 64-bit versions of Windows available; before this, there were only 32-bit versions of these operating systems.

Programs and subsystems in user mode are limited in terms of what system resources they have access to, while the kernel mode has unrestricted access to the system memory and external devices. Kernel mode in Windows NT has full access to the hardware and system resources of the computer. The Windows NT kernel is a hybrid kernel; the architecture comprises a simple kernel, hardware abstraction layer (HAL), drivers, and a range of services (collectively named Executive), which all exist in kernel mode.

User mode in Windows NT is made of subsystems capable of passing I/O requests to the appropriate kernel mode device drivers by using the I/O manager. The user mode layer of Windows NT is made up of the "Environment subsystems", which run applications written for many different types of operating systems, and the "Integral subsystem", which operates system-specific functions on behalf of environment subsystems. The kernel mode stops user mode services and applications from accessing critical areas of the operating system that they should not have access to.

The Executive interfaces, with all the user mode subsystems, deal with I/O, object management, security and process management. The kernel sits between the hardware abstraction layer and the Executive to provide multiprocessor synchronization, thread and interrupt scheduling and dispatching, and trap handling and exception dispatching. The kernel is also responsible for initializing device drivers at bootup. Kernel mode drivers exist in three levels: highest level drivers, intermediate drivers and low-level drivers. Windows Driver Model (WDM) exists in the intermediate layer and was mainly designed to be binary and source compatible between Windows 98 and Windows 2000. The lowest level drivers are either legacy Windows NT device drivers that control a device directly or can be a plug and play (PnP) hardware bus.

User mode

User mode is made up of various system-defined processes and DLLs.

The interface between user mode applications and operating system kernel functions is called an "environment subsystem." Windows NT can have more than one of these, each implementing a different API set. This mechanism was designed to support applications written for many different types of operating systems. None of the environment subsystems can directly access hardware; access to hardware functions is done by calling into kernel mode routines.

There are three main environment subsystems: the Win32 subsystem, an OS/2 subsystem and a POSIX subsystem.

Win32 environment subsystem

The Win32 environment subsystem can run 32-bit Windows applications. It contains the console as well as text window support, shutdown and hard-error handling for all other environment subsystems. It also supports Virtual DOS Machines (VDMs), which allow MS-DOS and 16-bit Windows (Win16) applications to run on Windows NT. There is a specific MS-DOS VDM that runs in its own address space and which emulates an Intel 80486 running MS-DOS 5.0. Win16 programs, however, run in a Win16 VDM. Each program, by default, runs in the same process, thus using the same address space, and the Win16 VDM gives each program its own thread on which to run. However, Windows NT does allow users to run a Win16 program in a separate Win16 VDM, which allows the program to be preemptively multitasked, as Windows NT will pre-empt the whole VDM process, which only contains one running application. The Win32 environment subsystem process (csrss.exe) also includes the window management functionality, sometimes called a "window manager". It handles input events (such as from the keyboard and mouse), then passes messages to the applications that need to receive this input. Each application is responsible for drawing or refreshing its own windows and menus, in response to these messages.

OS/2 environment subsystem

The OS/2 environment subsystem supports 16-bit character-based OS/2 applications and emulates OS/2 1.x, but not 32-bit or graphical OS/2 applications as used with OS/2 2.x or later, on x86 machines only. To run graphical OS/2 1.x programs, the Windows NT Add-On Subsystem for Presentation Manager must be installed. The last version of Windows NT to have an OS/2 subsystem was Windows 2000; it has been discontinued as of Windows XP.

POSIX environment subsystem

The POSIX environment subsystem supports applications that are strictly written to either the POSIX.1 standard or the related ISO/IEC standards. This subsystem has been replaced by Interix, which is a part of Windows Services for UNIX. This was in turn replaced by the Windows Subsystem for Linux.

Security subsystem

The security subsystem deals with security tokens, grants or denies access to user accounts based on resource permissions, handles login requests and initiates login authentication, and determines which system resources need to be audited by Windows NT. It also looks after Active Directory. The workstation service implements the network redirector, which is the client side of Windows file and print sharing; it implements local requests to remote files and printers by "redirecting" them to the appropriate servers on the network. Conversely, the server service allows other computers on the network to access file shares and shared printers offered by the local system.

Kernel mode

Windows NT kernel mode has full access to the hardware and system resources of the computer and runs code in a protected memory area. It controls access to scheduling, thread prioritization, memory management and the interaction with hardware. The kernel mode stops user mode services and applications from accessing critical areas of the operating system that they should not have access to; user mode processes must ask the kernel mode to perform such operations on their behalf.

While the x86 architecture supports four different privilege levels (numbered 0 to 3), only the two extreme privilege levels are used. Usermode programs are run with CPL 3, and the kernel runs with CPL 0. These two levels are often referred to as "ring 3" and "ring 0", respectively. Such a design decision had been done to achieve code portability to RISC platforms that only support two privilege levels, though this breaks compatibility with OS/2 applications that contain I/O privilege segments that attempt to directly access hardware.

Code running in kernel mode includes: the executive, which is itself made up of many modules that do specific tasks; the kernel, which provides low-level services used by the Executive; the Hardware Abstraction Layer (HAL); and kernel drivers.

Executive

The Windows Executive services make up the low-level kernel-mode portion, and are contained in the file NTOSKRNL.EXE. It deals with I/O, object management, security and process management. These are divided into several subsystems, among which are Cache Manager, Configuration Manager, I/O Manager, Local Procedure Call (LPC), Memory Manager, Object Manager, Process Structure and Security Reference Monitor (SRM). Grouped together, the components can be called Executive services (internal name Ex). System Services (internal name Nt), i.e., system calls, are implemented at this level, too, except very few that call directly into the kernel layer for better performance.

The term "service" in this context generally refers to a callable routine, or set of callable routines. This is distinct from the concept of a "service process", which is a user mode component somewhat analogous to a daemon in Unix-like operating systems.

Object Manager: The Object Manager (internal name Ob) is an executive subsystem that all other executive subsystems, especially system calls, must pass through to gain access to Windows NT resources—essentially making it a resource management infrastructure service. The object manager is used to reduce the duplication of object resource management functionality in other executive subsystems, which could potentially lead to bugs and make development of Windows NT harder. To the object manager, each resource is an object, whether that resource is a physical resource (such as a file system or peripheral) or a logical resource (such as a file). Each object has a structure or object type that the object manager must know about.; Object creation is a process in two phases, creation and insertion. Creation causes the allocation of an empty object and the reservation of any resources required by the object manager, such as an (optional) name in the namespace. If creation was successful, the subsystem responsible for the creation fills in the empty object. Finally, if the subsystem deems the initialization successful, it instructs the object manager to insert the object, which makes it accessible through its (optional) name or a cookie called a handle. From then on, the lifetime of the object is handled by the object manager, and it's up to the subsystem to keep the object in a working condition until being signaled by the object manager to dispose of it.; Handles are identifiers that represent a reference to a kernel resource through an opaque value. Similarly, opening an object through its name is subject to security checks, but acting through an existing, open handle is only limited to the level of access requested when the object was opened or created.; Object types define the object procedures and any data specific to the object. In this way, the object manager allows Windows NT to be an object-oriented operating system, as object types can be thought of as polymorphic classes that define objects. Most subsystems, though, with a notable exception in the I/O Manager, rely on the default implementation for all object type procedures.; Each instance of an object that is created stores its name, parameters that are passed to the object creation function, security attributes and a pointer to its object type. The object also contains an object close procedure and a reference count to tell the object manager how many other objects in the system reference that object and thereby determines whether the object can be destroyed when a close request is sent to it. Every named object exists in a hierarchical object namespace.
Cache Controller: Closely coordinates with the Memory Manager, I/O Manager and I/O drivers to provide a common cache for regular file I/O. The Windows Cache Manager operates on file blocks (rather than device blocks), for consistent operation between local and remote files, and ensures a certain degree of coherency with memory-mapped views of files, since cache blocks are a special case of memory-mapped views and cache misses a special case of page faults.
Configuration Manager: Implements the system calls needed by Windows Registry.
I/O Manager: Allows devices to communicate with user-mode subsystems. It translates user-mode read and write commands into read or write IRPs which it passes to device drivers. It accepts file system I/O requests and translates them into device specific calls, and can incorporate low-level device drivers that directly manipulate hardware to either read input or write output. It also includes a cache manager to improve disk performance by caching read requests and write to the disk in the background.
Local Procedure Call (LPC): Provides inter-process communication ports with connection semantics. LPC ports are used by user-mode subsystems to communicate with their clients, by Executive subsystems to communicate with user-mode subsystems, and as the basis for the local transport for Microsoft RPC.
Memory Manager: Manages virtual memory, controlling memory protection and the paging of memory in and out of physical memory to secondary storage, and implements a general-purpose allocator of physical memory. It also implements a parser of PE executables that lets an executable be mapped or unmapped in a single, atomic step.; Starting from Windows NT Server 4.0, Terminal Server Edition, the memory manager implements a so-called session space, a range of kernel-mode memory that is subject to context switching just like user-mode memory. This lets multiple instances of the kernel-mode Win32 subsystem and GDI drivers run side-by-side, despite shortcomings in their initial design. Each session space is shared by several processes, collectively referred to as a "session".; To ensure a degree of isolation between sessions without introducing a new object type, the association between processes and sessions is handled by the Security Reference Monitor, as an attribute of a security subject (token), and it can only be changed while holding special privileges.; The relatively unsophisticated and ad hoc nature of sessions is due to the fact they weren't part of the initial design, and had to be developed, with minimal disruption to the main line, by a third party (Citrix Systems) as a prerequisite for their terminal server product for Windows NT, called WinFrame. Starting with Windows Vista, though, sessions finally became a proper aspect of the Windows architecture. No longer a memory manager construct that creeps into user mode indirectly through Win32, they were expanded into a pervasive abstraction affecting most Executive subsystems. As a matter of fact, regular use of Windows Vista always results in a multi-session environment.
Process Structure: Handles process and thread creation and termination, and it implements the concept of Job, a group of processes that can be terminated as a whole, or be placed under shared restrictions (such as the total maximum of allocated memory, or CPU time). Job objects were introduced in Windows 2000.
PnP Manager: Handles plug and play and supports device detection and installation at boot time. It also has the responsibility to stop and start devices on demand—this can happen when a bus (such as USB or IEEE 1394 FireWire) gains a new device and needs to have a device driver loaded to support it. Its bulk is actually implemented in user mode, in the Plug and Play Service, which handles the often complex tasks of installing the appropriate drivers, notifying services and applications of the arrival of new devices, and displaying GUI to the user.
Power Manager: Deals with power events (power-off, stand-by, hibernate, etc.) and notifies affected drivers with special IRPs (Power IRPs).
Security Reference Monitor (SRM): The primary authority for enforcing the security rules of the security integral subsystem. It determines whether an object or resource can be accessed, via the use of access control lists (ACLs), which are themselves made up of access control entries (ACEs). ACEs contain a Security Identifier (SID) and a list of operations that the ACE gives a select group of trustees—a user account, group account, or login session—permission (allow, deny, or audit) to that resource.
GDI: The Graphics Device Interface is responsible for tasks such as drawing lines and curves, rendering fonts and handling palettes. The Windows NT 3.x series of releases had placed the GDI component in the user-mode Client/Server Runtime Subsystem, but this was moved into kernel mode with Windows NT 4.0 to improve graphics performance.

Kernel

The kernel sits between the HAL and the Executive and provides multiprocessor synchronization, thread and interrupt scheduling and dispatching, and trap handling and exception dispatching; it is also responsible for initializing device drivers at bootup that are necessary to get the operating system up and running. That is, the kernel performs almost all the tasks of a traditional microkernel; the strict distinction between Executive and Kernel is the most prominent remnant of the original microkernel design, and historical design documentation consistently refers to the kernel component as "the microkernel".

The kernel often interfaces with the process manager. The level of abstraction is such that the kernel never calls into the process manager, only the other way around (save for a handful of corner cases, still never to the point of a functional dependence).

Hybrid kernel design

The Windows NT design includes many of the same objectives as Mach, the archetypal microkernel system, one of the most important being its structure as a collection of modules that communicate via well-known interfaces, with a small microkernel limited to core functions such as first-level interrupt handling, thread scheduling and synchronization primitives. This allows for the possibility of using either direct procedure calls or interprocess communication (IPC) to communicate between modules, and hence for the potential location of modules in different address spaces (for example in either kernel space or server processes). Other design goals shared with Mach included support for diverse architectures, a kernel with abstractions general enough to allow multiple operating system personalities to be implemented on top of it and an object-oriented organisation.

The primary operating system personality on Windows is the Windows API, which is always present. The emulation subsystem which implements the Windows personality is called the Client/Server Runtime Subsystem (csrss.exe). On versions of NT prior to 4.0, this subsystem process also contained the window manager, graphics device interface and graphics device drivers. For performance reasons, however, in version 4.0 and later, these modules (which are often implemented in user mode even on monolithic systems, especially those designed without internal graphics support) run as a kernel-mode subsystem.

Applications that run on NT are written to one of the OS personalities (usually the Windows API), and not to the native NT API for which documentation is not publicly available (with the exception of routines used in device driver development). An OS personality is implemented via a set of user-mode DLLs (see Dynamic-link library), which are mapped into application processes' address spaces as required, together with an emulation subsystem server process (as described previously). Applications access system services by calling into the OS personality DLLs mapped into their address spaces, which in turn call into the NT run-time library (ntdll.dll), also mapped into the process address space. The NT run-time library services these requests by trapping into kernel mode to either call kernel-mode Executive routines or make Local Procedure Calls (LPCs) to the appropriate user-mode subsystem server processes, which in turn use the NT API to communicate with application processes, the kernel-mode subsystems and each other.

Kernel-mode drivers

Windows NT uses kernel-mode device drivers to enable it to interact with hardware devices. Each of the drivers has well defined system routines and internal routines that it exports to the rest of the operating system. All devices are seen by user mode code as a file object in the I/O manager, though to the I/O manager itself the devices are seen as device objects, which it defines as either file, device or driver objects. Kernel mode drivers exist in three levels: highest level drivers, intermediate drivers and low level drivers. The highest level drivers, such as file system drivers for FAT and NTFS, rely on intermediate drivers. Intermediate drivers consist of function drivers—or main driver for a device—that are optionally sandwiched between lower and higher level filter drivers. The function driver then relies on a bus driver—or a driver that services a bus controller, adapter, or bridge—which can have an optional bus filter driver that sits between itself and the function driver. Intermediate drivers rely on the lowest level drivers to function. The Windows Driver Model (WDM) exists in the intermediate layer. The lowest level drivers are either legacy Windows NT device drivers that control a device directly or can be a PnP hardware bus. These lower level drivers directly control hardware and do not rely on any other drivers.

Hardware abstraction layer

The Windows NT hardware abstraction layer (HAL) is a layer between the physical hardware of the computer and the rest of the operating system. It was designed to hide differences in hardware and provide a consistent platform on which the kernel is run. The HAL includes hardware-specific code that controls I/O interfaces, interrupt controllers and multiple processors.

However, despite its purpose and designated place within the architecture, the HAL isn't a layer that sits entirely below the kernel, the way the kernel sits below the Executive: All known HAL implementations depend in some measure on the kernel, or even the Executive. In practice, this means that kernel and HAL variants come in matching sets that are specifically constructed to work together.

In particular hardware abstraction does not involve abstracting the instruction set, which generally falls under the wider concept of portability. Abstracting the instruction set, when necessary (such as for handling the several revisions to the x86 instruction set, or emulating a missing math coprocessor), is performed by the kernel, or via hardware virtualization.

Boot sequence

The boot sequence is initiated by NTLDR in versions before Vista and the Windows Boot Manager in Vista and later. The boot loader is responsible for accessing the file system on the boot drive, starting ntoskrnl.exe, and loading boot-time device drivers into memory. Once all the boot and system drivers have been loaded, the kernel starts the Session Manager Subsystem. The session manager starts crucial kernel and user mode services of the Win32 subsystem, such as the Client/Server Runtime Subsystem. The session also runs process winlogon, allowing the users to login and use their accounts.

Object-oriented operating system

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Object-oriented_operating_system

An object-oriented operating system is an operating system that is designed, structured, and operated using object-oriented programming principles.

An object-oriented operating system is in contrast to an object-oriented user interface or programming framework, which can be run on a non-object-oriented operating system like DOS or Unix.

There are already object-based language concepts involved in the design of a more typical operating system such as Unix. While a more traditional language like C does not support object-orientation as fluidly as more recent languages, the notion of, for example, a file, stream, or device driver (in Unix, each represented as a file descriptor) can be considered a good example of objects. They are, after all, abstract data types, with various methods in the form of system calls which behavior varies based on the type of object and which implementation details are hidden from the caller.

Object-orientation has been defined as objects + inheritance, and inheritance is only one approach to the more general problem of delegation that occurs in every operating system. Object-orientation has been more widely used in the user interfaces of operating systems than in their kernels.

Background

An object is an instance of a class, which provides a certain set of functionalities. Two objects can be differentiated based on the functionalities (or methods) they support. In an operating system context, objects are associated with a resource. Historically, the object-oriented design principles were used in operating systems to provide several protection mechanisms.

Protection mechanisms in an operating system help in providing a clear separation between different user programs. It also protects the operating system from any malicious user program behavior. For example, consider the case of user profiles in an operating system. The user should not have access to resources of another user. The object model deals with these protection issues with each resource acting as an object. Every object can perform only a set of operations. In the context of user profiles, the set of operations is limited by privilege level of a user.

Present-day operating systems use object-oriented design principles for many components of the system, which includes protection.

Examples

Athene

Athene is an object-based operating system first released in 2000 by Rocklyte Systems. The user environment was constructed entirely from objects that are linked together at runtime. Applications for Athene could also be created using this methodology and were commonly scripted using the object scripting language Dynamic Markup Language (DML). Objects could have been shared between processes by creating them in shared memory and locking them as needed for access. Athene's object framework was multi-platform, allowing it to be used in Windows and Linux environments for developing object-oriented programs. The company went defunct and the project abandoned sometime in 2009.

BeOS

BeOS was an object-oriented operating system released in 1995, which used objects and the C++ language for the application programming interface (API). The kernel was written in C with C++ wrappers in user space. The OS did not see mainstream usage and proved commercially unviable, however it has seen continued usage and development by a small enthusiast community.

Choices

Choices is an object-oriented operating system developed at the University of Illinois at Urbana–Champaign. It is written in C++ and uses objects to represent core kernel components like the central processing unit (CPU), processes, and so on. Inheritance is used to separate the kernel into portable machine-independent classes and small non-portable dependent classes. Choices has been ported to and runs on SPARC, x86, and ARM.

ETHOS

ETHOS was an experimental object oriented version of the Oberon System (see below) created by Clemens Szyperski for his PhD Thesis written in Oberon-2

GEOS

PC/GEOS is a light-weight object-oriented multitasking graphical operating system with sophisticated window and desktop management featuring scalable fonts. It is mostly written in an object-oriented x86 assembly language dialect and some C/C++ and is designed to run on DOS (similar to Microsoft Windows up to Windows Me). GEOS was developed originally by Berkeley Softworks in 1990, which later became GeoWorks Corporation, and it is continued to be maintained by BreadBox Computer Company. Related software suites were named Ensemble and New Deal Office. Adaptations exist for various palmtops, and 32-bit systems with non-x86-CPUs.

Haiku

Haiku (originally named OpenBeOS), is an open-source replacement for BeOS. It reached its first development milestone in September 2009 with the release of Haiku R1/Alpha 1. The x86 distribution is compatible with BeOS at both source and binary level. Like BeOS, it is written primarily in C++ and provides an object-oriented API. It is actively developed.

IBM i (OS/400, i5/OS)

IBM introduced OS/400 in 1988. This OS ran exclusively on the AS/400 platform. Renamed IBM i in 2008, this operating system and runs exclusively on Power Systems which also can run AIX and Linux. IBM i uses an object-oriented methodology and integrates a database (Db2 for i). The IBM i OS has a 128-bit unique identifier for each object.

IBM OS/2 2.0

IBM's first priority based pre-emptive multitasking, graphical, windows-based operating system included an object-oriented user shell. It was designed for the Intel 80386 that used virtual 8086 mode with full 32-bit support and was released in 1992. ArcaOS, a new OS/2 based operating system initially called Blue Lion^[9] is being developed by Arca Noae. The first version was released in May 2017.

IBM TopView

TopView was an object-oriented operating environment that loaded on a PC on DOS, and then took control from DOS. At that point it effectively became an object-oriented operating system with an object-oriented API (TopView API). It was IBM's first multi-tasking, window based, object-oriented operating system for the PC led by David C. Morrill and released in February 1985.

Java-based

Given that Oracle's (formerly Sun Microsystems') Java is today one of the most dominant object-oriented languages, it is no surprise that Java-based operating systems have been attempted. In this area, ideally, the kernel would consist of the bare minimum needed to support a Java virtual machine (JVM). This is the only component of such an operating system that would have to be written in a language other than Java. Built on the JVM and basic hardware support, it would be possible to write the rest of the operating system in Java; even parts of the system that are more traditionally written in a lower-level language such as C, for example device drivers, can be written in Java.

Examples of attempts at such an operating system include JavaOS, JOS, JNode, and JX.

Lisp-based

An object-oriented operating system written in the Lisp dialect Lisp Machine Lisp (and later Common Lisp) was developed at MIT. It was commercialized with Lisp machines from Symbolics, Lisp Machines Inc. and Texas Instruments. Symbolics called their operating system Genera. It was developed with the Flavors object-oriented extension of Lisp, then with New Flavors, and then with the Common Lisp Object System (CLOS).

Xerox developed several workstations with an operating system written in Interlisp-D. Interlisp-D provided object-oriented extensions like LOOPS and CLOS.

Movitz and Mezzano are two more recent attempts at operating systems written in Common Lisp.

Medos-2

Medos-2 is a single user, object-oriented operating system made for the Lilith line of workstations (processor: Advanced Micro Devices (AMD) 2901), developed in the early 1980s at ETH Zurich by Svend Erik Knudsen with advice from Niklaus Wirth. It is built entirely from modules of the programming language Modula-2.It was succeeded at ETH Zurich by the Oberon system (see also below), and a variant named Excelsior was developed for the Kronos workstation, by the Academy of Sciences of the Soviet Union, Siberian branch, Novosibirsk Computing Center, Modular Asynchronous Developable Systems (MARS) project, Kronos Research Group (KRG).

Microsoft Singularity

Singularity is an experimental operating system based on Microsoft's .NET Framework. It is comparable to Java-based operating systems.

Microsoft Windows NT

Windows NT is a family of operating systems (including Windows 7, 8, Phone 8, 8.1, Windows 10, 10 Mobile, Windows 11 and Xbox) produced by Microsoft, the first version of which was released in July 1993. It is a high-level programming language-based, processor-independent, multiprocessing, multi-user operating system. It is best described as object-based rather than object-oriented as it does not include the full inheritance properties of object-oriented languages.

The Object Manager is in charge of managing NT objects. As part of this responsibility, it maintains an internal namespace where various operating system components, device drivers, and Win32 programs can store and lookup objects. The NT Native API provides routines that allow user space (mode) programs to browse the namespace and query the status of objects located there, but the interfaces are undocumented. NT supports per-object (file, function, and role) access control lists allowing a rich set of security permissions to be applied to systems and services. WinObj is a Windows NT program that uses the NT Native API (provided by NTDLL.DLL) to access and display information on the NT Object Manager's name space.

Component Object Model: On the user mode side of Windows, the Component Object Model (COM) is an application binary interface standard for software components introduced by Microsoft in 1993. It is used to enable interprocess communication and dynamic object creation in a large range of programming languages. COM is the basis for several other Microsoft technologies and frameworks, including Object Linking and Embedding (OLE), OLE Automation, ActiveX, COM+, Distributed Component Object Model (DCOM), the Windows shell, DirectX, and Windows Runtime. OLE is a proprietary technology developed by Microsoft that allows embedding and linking to documents and other objects. On a technical level, an OLE object is any object that implements the IOleObject interface, possibly along with a wide range of other interfaces, depending on the object's needs. Its primary use is for managing compound documents, but it is also used for transferring data between different applications using drag and drop and clipboard operations.
Compound File Binary Format: Compound File Binary Format (CFBF) is a file format for storing many files and streams within one file on a disk. CFBF is developed by Microsoft and is an implementation of Microsoft COM Structured Storage. Structured storage is widely used as main file format in Microsoft Office applications including Microsoft Word, Microsoft Excel, Microsoft PowerPoint, Microsoft Access and is the basis of Advanced Authoring Format.
Object Linking and Embedding: Object Linking and Embedding (OLE) was part of a grander plan named Cairo, the code name for a project at Microsoft from 1991 to 1996. Its charter was to build technologies for a next generation operating system that would fulfill Bill Gates' vision of "information at your fingertips". Cairo never shipped, although parts of its technologies have since appeared in other products. The Windows 95 graphical user interface was based on the initial design work that was done on the Cairo user interface. The remaining component is the object-based file system. It was once planned to be implemented in the form of Windows Future Storage (WinFS) as part of Windows Vista. WinFS is the code name for data storage and management system project based on relational databases, running on the Windows NT file system (NTFS), which is object-oriented in that it can store the NT objects including its NT object identifier. Each NTFS object has an object identifier; a shortcut with a target that's on an NTFS volume also records the object identifier of the shortcut target, and the object identifier of the drive. WinFS was first demonstrated in 2003 as an advanced storage subsystem for the Microsoft Windows operating system, designed for persistence and management of structured, semi-structured, and unstructured data. WinFS development was cancelled in June 2006, with some of its technologies merged into other Microsoft products such as Microsoft SQL Server 2008 and Microsoft SharePoint. It was subsequently confirmed in an interview with Bill Gates that Microsoft planned to migrate applications like Windows Media Player, Windows Photo Gallery, Microsoft Office Outlook, etc., to use WinFS as the data storage back-end.

NeXTSTEP: During the late 1980s, Steve Jobs formed the computer company NeXT. One of NeXT's first tasks was to design an object-oriented operating system, NeXTSTEP. They did this by adding an object-oriented framework on Mach and BSD using the Objective-C language as a basis. It achieved a niche status in the computing market, notably used by Tim Berners-Lee developing the first implementation of the World Wide Web.; NeXTStep later evolved into OpenStep and the Cocoa API on macOS and iOS (iPadOS, watchOS).; OpenStep was provided as an API layer on many operating systems, namely HP-UX, NextStep, Solaris, and Windows.
Oberon System: Oberon System is a single user, object-oriented operating system made for the Ceres line of workstations (processor: National Semiconductor NS32000), developed in the later 1980s at ETH Zurich by Niklaus Wirth and Jürg Gutknecht. It is built entirely from modules of the programming language Oberon. There are two successors of the Oberon System, ETHOS (see above) and, as of 2023 still maintained, an evolution named Active Object System (AOS), then renamed Bluebottle, then renamed A2.
OOSMOS: The Object-Oriented State Machine Operating System (OOSMOS), written in C, promotes object-oriented encapsulation and implements a full table-driven hierarchical state machine architecture. It generates C code directly from state charts drawn with the open source tool UMLet. OOSMOS also supports a unique feature call 'state threads' which allows a thread of execution per state. OOSMOS operates on a bare board or in cooperation with an existing operating system.
Phantom OS: Phantom OS adheres to a principle where "everything is an object" and eliminates the concept of a file entirely, instead transparently persisting virtual memory to storage.
ReactOS: ReactOS is an open-source operating system intended to be binary compatible with application software and device drivers made for Microsoft Windows NT versions. Written from scratch, it aims to follow the architecture of Windows NT designed by Microsoft from the hardware level right through to the application level. This is not a Linux-based system, and shares none of the unix architecture.
Smalltalk: Smalltalk was invented at Xerox in the 1970s. The Smalltalk system is fully object-oriented and needs very little support by BIOS and the run-time system.
Syllable: Syllable makes heavy use of C++ and for that reason is often compared to BeOS.
Symbolics Genera: Genera from Symbolics is an operating system for Lisp machines written in ZetaLisp and Symbolics Common Lisp. It makes heavy use of Flavors (an early object-oriented extension to Lisp) and the Common Lisp Object System (CLOS). Development began in the mid 70s at MIT.
Taligent: Taligent was an object-oriented operating system project, begun by Apple Inc. and jointly developed with IBM in the 1990s. It was later spun off to an IBM subsidiary and transformed from an operating system to a programming environment.

Software engineering

Software engineering is a branch of both computer science and engineering focused on designing, developing, testing, and maintaining software applications. It involves applying engineering principles and computer programming expertise to develop software systems that meet user needs.

The terms programmer and coder overlap software engineer, but they imply only the construction aspect of a typical software engineer workload.

A software engineer applies a software development process, which involves defining, implementing, testing, managing, and maintaining software systems, as well as developing the software development process itself.

History

Beginning in the 1960s, software engineering was recognized as a separate field of engineering.

The development of software engineering was seen as a struggle. Problems included software that was over budget, exceeded deadlines, required extensive debugging and maintenance, and unsuccessfully met the needs of consumers or was never even completed.

In 1968, NATO held the first software engineering conference, where issues related to software were addressed. Guidelines and best practices for the development of software were established.

The origins of the term software engineering have been attributed to various sources. The term appeared in a list of services offered by companies in the June 1965 issue of "Computers and Automation" and was used more formally in the August 1966 issue of Communications of the ACM (Volume 9, number 8) in "President's Letter to the ACM Membership" by Anthony A. Oettinger. It is also associated with the title of a NATO conference in 1968 by Professor Friedrich L. Bauer. Margaret Hamilton described the discipline of "software engineering" during the Apollo missions to give what they were doing legitimacy. At the time, there was perceived to be a "software crisis".The 40th International Conference on Software Engineering (ICSE 2018) celebrates 50 years of "Software Engineering" with the Plenary Sessions' keynotes of Frederick Brooks and Margaret Hamilton.

In 1984, the Software Engineering Institute (SEI) was established as a federally funded research and development center headquartered on the campus of Carnegie Mellon University in Pittsburgh, Pennsylvania, United States. Watts Humphrey founded the SEI Software Process Program, aimed at understanding and managing the software engineering process. The Process Maturity Levels introduced became the Capability Maturity Model Integration for Development (CMMI-DEV), which defined how the US Government evaluates the abilities of a software development team.

Modern, generally accepted best practices for software engineering have been collected by the ISO/IEC JTC 1/SC 7 subcommittee and published as the Software Engineering Body of Knowledge (SWEBOK). Software engineering is considered one of the major computing disciplines.

Terminology

Definition

Notable definitions of software engineering include:

"The systematic application of scientific and technological knowledge, methods, and experience to the design, implementation, testing, and documentation of software."—The Bureau of Labor Statistics—IEEE Systems and software engineering – Vocabulary
"The application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software."—IEEE Standard Glossary of Software Engineering Terminology
"An engineering discipline that is concerned with all aspects of software production."—Ian Sommerville
"The establishment and use of sound engineering principles in order to economically obtain software that is reliable and works efficiently on real machines."—Fritz Bauer
"A branch of computer science that deals with the design, implementation, and maintenance of complex computer programs."—Merriam-Webster
"'Software engineering' encompasses not just the act of writing code, but all of the tools and processes an organization uses to build and maintain that code over time. [...] Software engineering can be thought of as 'programming integrated over time.'"—Software Engineering at Google

The term has also been used less formally:

as the informal contemporary term for the broad range of activities that were formerly called computer programming and systems analysis
as the broad term for all aspects of the practice of computer programming, as opposed to the theory of computer programming, which is formally studied as a sub-discipline of computer science
as the term embodying the advocacy of a specific approach to computer programming, one that urges that it be treated as an engineering discipline rather than an art or a craft, and advocates the codification of recommended practices

Suitability

Individual commentators have disagreed sharply on how to define software engineering or its legitimacy as an engineering discipline. David Parnas has said that software engineering is, in fact, a form of engineering. Steve McConnell has said that it is not, but that it should be. Donald Knuth has said that programming is an art and a science. Edsger W. Dijkstra claimed that the terms software engineering and software engineer have been misused in the United States.

Workload

Requirements analysis

Requirements engineering is about elicitation, analysis, specification, and validation of requirements for software. Software requirements can be functional, non-functional or domain.

Functional requirements describe expected behaviors (i.e. outputs). Non-functional requirements specify issues like portability, security, maintainability, reliability, scalability, performance, reusability, and flexibility. They are classified into the following types: interface constraints, performance constraints (such as response time, security, storage space, etc.), operating constraints, life cycle constraints (maintainability, portability, etc.), and economic constraints. Knowledge of how the system or software works is needed when it comes to specifying non-functional requirements. Domain requirements have to do with the characteristic of a certain category or domain of projects.

Design

Software design is the process of making high-level plans for the software. Design is sometimes divided into levels:

Interface design plans the interaction between a system and its environment as well as the inner workings of the system.
Architectural design plans the major components of a system, including their responsibilities, properties, and interfaces between them.
Detailed design plans internal elements, including their properties, relationships, algorithms and data structures.

Construction

Software construction typically involves programming (a.k.a. coding), unit testing, integration testing, and debugging so as to implement the design."Software testing is related to, but different from, ... debugging". Testing during this phase is generally performed by the programmer and with the purpose to verify that the code behaves as designed and to know when the code is ready for the next level of testing.

Testing

Software testing is an empirical, technical investigation conducted to provide stakeholders with information about the quality of the software under test.

When described separately from construction, testing typically is performed by test engineers or quality assurance instead of the programmers who wrote it. It is performed at the system level and is considered an aspect of software quality.

Program analysis

Program analysis is the process of analyzing computer programs with respect to an aspect such as performance, robustness, and security.

Maintenance

Software maintenance refers to supporting the software after release. It may include but is not limited to: error correction, optimization, deletion of unused and discarded features, and enhancement of existing features.

Usually, maintenance takes up 40% to 80% of project cost.

Education

Knowledge of computer programming is a prerequisite for becoming a software engineer. In 2004, the IEEE Computer Society produced the SWEBOK, which has been published as ISO/IEC Technical Report 1979:2005, describing the body of knowledge that they recommend to be mastered by a graduate software engineer with four years of experience. Many software engineers enter the profession by obtaining a university degree or training at a vocational school. One standard international curriculum for undergraduate software engineering degrees was defined by the Joint Task Force on Computing Curricula of the IEEE Computer Society and the Association for Computing Machinery, and updated in 2014. A number of universities have Software Engineering degree programs; as of 2010, there were 244 Campus Bachelor of Software Engineering programs, 70 Online programs, 230 Masters-level programs, 41 Doctorate-level programs, and 69 Certificate-level programs in the United States.

In addition to university education, many companies sponsor internships for students wishing to pursue careers in information technology. These internships can introduce the student to real-world tasks that typical software engineers encounter every day. Similar experience can be gained through military service in software engineering.

Software engineering degree programs

Half of all practitioners today have degrees in computer science, information systems, or information technology. A small but growing number of practitioners have software engineering degrees. In 1987, the Department of Computing at Imperial College London introduced the first three-year software engineering bachelor's degree in the world; in the following year, the University of Sheffield established a similar program. In 1996, the Rochester Institute of Technology established the first software engineering bachelor's degree program in the United States; however, it did not obtain ABET accreditation until 2003, the same year as Rice University, Clarkson University, Milwaukee School of Engineering, and Mississippi State University. In 1997, PSG College of Technology in Coimbatore, India was the first to start a five-year integrated Master of Science degree in Software Engineering.

Since then, software engineering undergraduate degrees have been established at many universities. A standard international curriculum for undergraduate software engineering degrees, SE2004, was defined by a steering committee between 2001 and 2004 with funding from the Association for Computing Machinery and the IEEE Computer Society. As of 2004, about 50 universities in the U.S. offer software engineering degrees, which teach both computer science and engineering principles and practices. The first software engineering master's degree was established at Seattle University in 1979. Since then, graduate software engineering degrees have been made available from many more universities. Likewise in Canada, the Canadian Engineering Accreditation Board (CEAB) of the Canadian Council of Professional Engineers has recognized several software engineering programs.

In 1998, the US Naval Postgraduate School (NPS) established the first doctorate program in Software Engineering in the world. Additionally, many online advanced degrees in Software Engineering have appeared such as the Master of Science in Software Engineering (MSE) degree offered through the Computer Science and Engineering Department at California State University, Fullerton. Steve McConnell opines that because most universities teach computer science rather than software engineering, there is a shortage of true software engineers. ETS (École de technologie supérieure) University and UQAM (Université du Québec à Montréal) were mandated by IEEE to develop the Software Engineering Body of Knowledge (SWEBOK), which has become an ISO standard describing the body of knowledge covered by a software engineer.

Profession

Legal requirements for the licensing or certification of professional software engineers vary around the world. In the UK, there is no licensing or legal requirement to assume or use the job title Software Engineer. In some areas of Canada, such as Alberta, British Columbia, Ontario, and Quebec, software engineers can hold the Professional Engineer (P.Eng) designation and/or the Information Systems Professional (I.S.P.) designation. In Europe, Software Engineers can obtain the European Engineer (EUR ING) professional title. Software Engineers can also become professionally qualified as a Chartered Engineer through the British Computer Society.

In the United States, the NCEES began offering a Professional Engineer exam for Software Engineering in 2013, thereby allowing Software Engineers to be licensed and recognized. NCEES ended the exam after April 2019 due to lack of participation. Mandatory licensing is currently still largely debated, and perceived as controversial.

The IEEE Computer Society and the ACM, the two main US-based professional organizations of software engineering, publish guides to the profession of software engineering. The IEEE's Guide to the Software Engineering Body of Knowledge – 2004 Version, or SWEBOK, defines the field and describes the knowledge the IEEE expects a practicing software engineer to have. The most current version is SWEBOK v4. The IEEE also promulgates a "Software Engineering Code of Ethics".

Employment

There are an estimated 26.9 million professional software engineers in the world as of 2022, up from 21 million in 2016.

Many software engineers work as employees or contractors. Software engineers work with businesses, government agencies (civilian or military), and non-profit organizations. Some software engineers work for themselves as freelancers. Some organizations have specialists to perform each of the tasks in the software development process. Other organizations require software engineers to do many or all of them. In large projects, people may specialize in only one role. In small projects, people may fill several or all roles at the same time. Many companies hire interns, often university or college students during a summer break, or externships. Specializations include analysts, architects, developers, testers, technical support, middleware analysts, project managers, software product managers, educators, and researchers.

Most software engineers and programmers work 40 hours a week, but about 15 percent of software engineers and 11 percent of programmers worked more than 50 hours a week in 2008. Potential injuries in these occupations are possible because like other workers who spend long periods sitting in front of a computer terminal typing at a keyboard, engineers and programmers are susceptible to eyestrain, back discomfort, Thrombosis, Obesity, and hand and wrist problems such as carpal tunnel syndrome.

United States

The U. S. Bureau of Labor Statistics (BLS) counted 1,365,500 software developers holding jobs in the U.S. in 2018. Due to its relative newness as a field of study, formal education in software engineering is often taught as part of a computer science curriculum, and many software engineers hold computer science degrees. The BLS estimates from 2023 to 2033 that computer software engineering would increase by 17%. This is down from the 2022 to 2032 BLS estimate of 25% for software engineering.And, is further down from their 30% 2010 to 2020 BLS estimate. Due to this trend, job growth may not be as fast as during the last decade, as jobs that would have gone to computer software engineers in the United States would instead be outsourced to computer software engineers in countries such as India and other foreign countries. In addition, the BLS Job Outlook for Computer Programmers, the U.S. Bureau of Labor Statistics (BLS) Occupational Outlook predicts a decline of -7 percent from 2016 to 2026, a further decline of -9 percent from 2019 to 2029, a decline of -10 percent from 2021 to 2031. and then a decline of -11 percent from 2022 to 2032. Since computer programming can be done from anywhere in the world, companies sometimes hire programmers in countries where wages are lower. Furthermore, the ratio of women in many software fields has also been declining over the years as compared to other engineering fields. Then there is the additional concern that recent advances in Artificial Intelligence might impact the demand for future generations of Software Engineers. However, this trend may change or slow in the future as many current software engineers in the U.S. market flee the profession or age out of the market in the next few decades.

Certification

The Software Engineering Institute offers certifications on specific topics like security, process improvement and software architecture. IBM, Microsoft and other companies also sponsor their own certification examinations. Many IT certification programs are oriented toward specific technologies, and managed by the vendors of these technologies. These certification programs are tailored to the institutions that would employ people who use these technologies.

Broader certification of general software engineering skills is available through various professional societies. As of 2006, the IEEE had certified over 575 software professionals as a Certified Software Development Professional (CSDP). In 2008 they added an entry-level certification known as the Certified Software Development Associate (CSDA). The ACM had a professional certification program in the early 1980s, which was discontinued due to lack of interest. The ACM and the IEEE Computer Society together examined the possibility of licensing of software engineers as Professional Engineers in the 1990s, but eventually decided that such licensing was inappropriate for the professional industrial practice of software engineering. John C. Knight and Nancy G. Leveson presented a more balanced analysis of the licensing issue in 2002.

In the U.K. the British Computer Society has developed a legally recognized professional certification called Chartered IT Professional (CITP), available to fully qualified members (MBCS). Software engineers may be eligible for membership of the British Computer Society or Institution of Engineering and Technology and so qualify to be considered for Chartered Engineer status through either of those institutions. In Canada the Canadian Information Processing Society has developed a legally recognized professional certification called Information Systems Professional (ISP). In Ontario, Canada, Software Engineers who graduate from a Canadian Engineering Accreditation Board (CEAB) accredited program, successfully complete PEO's (Professional Engineers Ontario) Professional Practice Examination (PPE) and have at least 48 months of acceptable engineering experience are eligible to be licensed through the Professional Engineers Ontario and can become Professional Engineers P.Eng. The PEO does not recognize any online or distance education however; and does not consider Computer Science programs to be equivalent to software engineering programs despite the tremendous overlap between the two. This has sparked controversy and a certification war. It has also held the number of P.Eng holders for the profession exceptionally low. The vast majority of working professionals in the field hold a degree in CS, not SE. Given the difficult certification path for holders of non-SE degrees, most never bother to pursue the license.

Impact of globalization

The initial impact of outsourcing, and the relatively lower cost of international human resources in developing third world countries led to a massive migration of software development activities from corporations in North America and Europe to India and later: China, Russia, and other developing countries. This approach had some flaws, mainly the distance / time zone difference that prevented human interaction between clients and developers and the massive job transfer. This had a negative impact on many aspects of the software engineering profession. For example, some students in the developed world avoid education related to software engineering because of the fear of offshore outsourcing (importing software products or services from other countries) and of being displaced by foreign visa workers. Although statistics do not currently show a threat to software engineering itself; a related career, computer programming does appear to have been affected. Nevertheless, the ability to smartly leverage offshore and near-shore resources via the follow-the-sun workflow has improved the overall operational capability of many organizations. When North Americans leave work, Asians are just arriving to work. When Asians are leaving work, Europeans arrive to work. This provides a continuous ability to have human oversight on business-critical processes 24 hours per day, without paying overtime compensation or disrupting a key human resource, sleep patterns.

While global outsourcing has several advantages, global – and generally distributed – development can run into serious difficulties resulting from the distance between developers. This is due to the key elements of this type of distance that have been identified as geographical, temporal, cultural and communication (that includes the use of different languages and dialects of English in different locations). Research has been carried out in the area of global software development over the last 15 years and an extensive body of relevant work published that highlights the benefits and problems associated with the complex activity. As with other aspects of software engineering research is ongoing in this and related areas.

Prizes

There are various prizes in the field of software engineering:

ACM-AAAI Allen Newell Award- USA. Awarded to career contributions that have breadth within computer science, or that bridge computer science and other disciplines.
BCS Lovelace Medal. Awarded to individuals who have made outstanding contributions to the understanding or advancement of computing.
ACM SIGSOFT Outstanding Research Award, selected for individual(s) who have made "significant and lasting research contributions to the theory or practice of software engineering."
More ACM SIGSOFT Awards.
The Codie award, a yearly award issued by the Software and Information Industry Association for excellence in software development within the software industry.
Harlan Mills Award for "contributions to the theory and practice of the information sciences, focused on software engineering".
ICSE Most Influential Paper Award.
Jolt Award, also for the software industry.
Stevens Award given in memory of Wayne Stevens.

Criticism

Some call for licensing, certification and codified bodies of knowledge as mechanisms for spreading the engineering knowledge and maturing the field.

Some claim that the concept of software engineering is so new that it is rarely understood, and it is widely misinterpreted, including in software engineering textbooks, papers, and among the communities of programmers and crafters.

Some claim that a core issue with software engineering is that its approaches are not empirical enough because a real-world validation of approaches is usually absent, or very limited and hence software engineering is often misinterpreted as feasible only in a "theoretical environment."

Edsger Dijkstra, a founder of many of the concepts in software development today, rejected the idea of "software engineering" up until his death in 2002, arguing that those terms were poor analogies for what he called the "radical novelty" of computer science:

A number of these phenomena have been bundled under the name "Software Engineering". As economics is known as "The Miserable Science", software engineering should be known as "The Doomed Discipline", doomed because it cannot even approach its goal since its goal is self-contradictory. Software engineering, of course, presents itself as another worthy cause, but that is eyewash: if you carefully read its literature and analyse what its devotees actually do, you will discover that software engineering has accepted as its charter "How to program if you cannot."

Search This Blog