File and I/O Systems

The File System is a crucial part of any operating system. It's responsible for organizing, storing, and retrieving data on secondary storage devices like hard drives, SSDs, and USB drives. Think of it as a librarian for your computer's data, keeping everything cataloged and easily accessible. Without a file system, your data would be a jumbled mess, and the OS wouldn't know where to find or save anything.

Core Concepts of File Systems

At its heart, a file system manages files and directories.

• Files: A file is a named collection of related information or data. It's the basic unit of storage. Files can contain anything from text documents and images to program executables and system configuration data.
• Directories (Folders): Directories are special files that contain other files and directories. They provide a hierarchical structure, allowing us to organize our files logically. This is similar to how you might organize documents in physical folders within a filing cabinet.
• File Attributes: Each file has associated attributes that provide information about it. These typically include:
  • Name: The identifier of the file.
  • Type: The kind of data the file contains (e.g., text, executable, image).
  • Location: The physical address of the file on the storage device.
  • Size: The amount of space the file occupies.
  • Protection: Access control information (who can read, write, execute).
  • Timestamps: Creation time, last access time, last modification time.
• File Operations: Users and the OS perform various operations on files, such as:
  • Create: Making a new file.
  • Delete: Removing a file.
  • Open: Preparing a file for access.
  • Close: Releasing a file after access.
  • Read: Copying data from the file to memory.
  • Write: Copying data from memory to the file.
  • Seek: Changing the current position within a file.

File System Implementation

Implementing a file system involves several layers, from the physical storage device up to the user interface.

1. Logical File System:

This is the part that users interact with. It manages file names, directories, and the mapping of logical file structures to physical blocks on the disk. It provides the abstraction of files and directories to the user and applications.

2. Basic File System:

This layer translates file operations into disk I/O requests. It deals with the physical block allocation and deallocation on the disk. It needs to keep track of which blocks are free and which are allocated to which files.

3. I/O Control System:

This layer consists of device drivers and interrupt handlers. It communicates directly with the hardware (disk controller) to perform the actual reading and writing of data blocks.

Directory Structures

The way directories are organized affects how users find and manage files.

• Single-Level Directory: All files are in one directory. Simple but prone to naming conflicts as the number of files grows.
• Two-Level Directory: Each user has their own directory. Files are unique within a user's directory. A master file directory (MFD) maps user names to user file directories (UFDs).
• Tree-Structured Directory: A hierarchical structure with a root directory. Each directory can contain files and other subdirectories. This is the most common approach (e.g., in Windows, Linux, macOS).
• Acyclic-Graph Directory: Allows directories to share files or subdirectories using links. This can lead to issues with file deletion (dangling pointers).
• General Graph Directory: The most flexible, allowing any directory to contain any other directory or file, including cycles. Complex to manage.

File Allocation Methods

These methods determine how disk space is allocated to files. The choice of method impacts performance, space utilization, and fragmentation.

• Contiguous Allocation: Each file occupies a contiguous block of disk space.
  • Pros: Simple to implement, excellent read/write performance (no seeking within a file).
  • Cons: Suffers from external fragmentation (free space is broken into small unusable chunks), difficult to grow files.
• Linked Allocation: Each file is a linked list of disk blocks. The directory entry contains a pointer to the first block. Each block contains a pointer to the next block.
  • Pros: Solves external fragmentation, easy to grow files.
  • Cons: Poor random access performance (must traverse the list), internal fragmentation (last block might not be full), link errors can cause data loss.
• Indexed Allocation: Each file has an index block that contains pointers to all the data blocks. The directory entry points to the index block.
  • Pros: Solves external fragmentation, good random access performance, easy to grow files.
  • Cons: Overhead of the index block (can be large for very large files), requires multiple disk accesses for an index block. Variations like linked index blocks or multi-level index blocks are used to handle large files more efficiently.

Free Space Management

The operating system needs to keep track of which blocks on the disk are free and available for allocation. Common methods include:

• Bitmap: A bit vector where each bit represents a block on the disk. 0 means free, 1 means allocated (or vice-versa). Simple but can be large for large disks.
• Linked List: A linked list of free blocks. Simple to implement but slow to find a contiguous chunk of free space.
• Grouping: A variation of the linked list where the first free block contains a list of other free blocks, and so on.
• Counting: Instead of just a bit, store a count of contiguous free blocks and the starting block number.

Disk Scheduling

When multiple I/O requests arrive, the disk scheduler determines the order in which to service them to optimize performance (minimize seek time and rotational latency) and throughput.

• First-Come, First-Served (FCFS): Service requests in the order they arrive. Simple but often inefficient.
• Shortest Seek Time First (SSTF): Service the request with the shortest seek time from the current head position. Can lead to starvation of requests further away.
• SCAN (Elevator Algorithm): The disk arm moves from one end of the disk to the other, servicing requests as it goes. When it reaches an end, it reverses direction.
• C-SCAN (Circular SCAN): Similar to SCAN, but when it reaches an end, it immediately jumps back to the beginning of the disk and starts scanning again, only servicing requests in one direction. Provides more uniform wait times.
• LOOK and C-LOOK: Variations of SCAN and C-SCAN where the arm only goes as far as the last request in each direction, rather than to the end of the disk.

I/O Hardware and Software

Input/Output (I/O) devices are how the computer interacts with the outside world. This includes everything from keyboards and mice to hard drives and network cards. The OS manages these devices through a layered I/O software structure.

I/O Hardware Components

• Devices: The actual peripheral devices (keyboard, disk, printer).
• Controller: Electronics that control the device. Each controller typically manages one or more devices. It contains logic for controlling the device and communicating with the CPU.
• Bus: A connection mechanism between the CPU, memory, and controllers.

I/O Software Layers

I/O software is typically structured in layers to provide a clean interface and manage complexity.

1. User-Level I/O Software: This includes the libraries (like C's `stdio.h`) that applications use for I/O. It provides a simplified, device-independent interface. For example, `printf()` works the same whether you're printing to the console or a file.
2. Device-Independent Software: This layer provides a uniform interface to the device drivers. It handles:
   • Buffering: Temporarily storing data to smooth out speed differences between CPU/memory and the device.
   • Error Reporting: Detecting and reporting errors from drivers.
   • Device Naming: Providing consistent names for devices.
   • Abstracting Device Characteristics: Hiding the specific details of each device type.
3. Device Drivers: This layer is device-specific. Each driver understands the specific hardware of a particular device or class of devices. It translates generic I/O requests from the device-independent layer into specific commands for the device controller.
4. Interrupt Handlers: When a device finishes an operation or needs attention, it sends an interrupt signal to the CPU. The interrupt handler is a piece of code that responds to this signal, acknowledges the interrupt, and potentially wakes up a waiting process.
5. Hardware: The actual device and controller.

I/O Techniques

The way the CPU interacts with I/O devices varies.

• Programmed I/O (PIO): The CPU is directly involved in every step of the I/O operation. It issues commands to the I/O device, waits for it to complete, and transfers data. This is simple but inefficient as it ties up the CPU.
• Interrupt-Driven I/O: The CPU issues an I/O command and then continues with other tasks. The device controller signals the CPU via an interrupt when the operation is complete. The CPU then handles the data transfer. This is more efficient than PIO.
• Direct Memory Access (DMA): A special hardware component (DMA controller) handles the data transfer directly between the I/O device and main memory, without involving the CPU. The CPU is only involved in setting up the DMA transfer and is notified via an interrupt when it's complete. This is the most efficient method for large data transfers.

Security and Protection Mechanisms

Security and protection are fundamental concerns in operating systems. They ensure that system resources are used appropriately and that unauthorized access or malicious activity is prevented.

Security Goals

• Confidentiality: Preventing unauthorized disclosure of information.
• Integrity: Ensuring that data is not modified in an unauthorized manner.
• Availability: Ensuring that authorized users can access resources when needed.
• Authentication: Verifying the identity of a user or process.
• Authorization: Determining what an authenticated user or process is allowed to do.

Protection Mechanisms

Protection refers to mechanisms within the OS that control the access of processes or users to the resources defined by the hardware and software.

• Domain of Protection: A domain specifies a set of objects (resources) together with the access rights that a process operating in that domain has to those objects. Domains can be defined by users, processes, or a combination.
• Access Control Lists (ACLs): For each object (file, device), maintain a list of subjects (users/processes) and the access rights they have to that object.
  • Example: File 'report.txt' might have an ACL: (Alice, Read, Write), (Bob, Read).
• Capabilities Lists: For each subject (user/process), maintain a list of objects and the access rights it possesses. This is the dual of ACLs.
• Principle of Least Privilege: Processes should be granted only the minimum privileges necessary to perform their tasks. This limits the damage if a process is compromised.

Authentication Methods

Verifying user identity is the first step in security.

• Passwords: The most common method. Security relies on keeping the password secret and securely storing password hashes.
• Biometrics: Fingerprints, facial recognition, iris scans.
• Smart Cards/Tokens: Physical devices that store authentication credentials.
• Multi-Factor Authentication (MFA): Requiring two or more different types of credentials (e.g., password + SMS code).

Operating System Security Features

• User Accounts and Groups: OS manages users and assigns them to groups, simplifying permission management.
• File Permissions: Standard Unix/Linux permissions (Read, Write, Execute for Owner, Group, Others) and Access Control Lists (ACLs) for finer-grained control.
• Process Isolation: Memory protection ensures one process cannot access the memory of another.
• Kernel Mode vs. User Mode: The CPU has modes to distinguish between privileged (kernel) and unprivileged (user) operations. Certain instructions can only be executed in kernel mode, protecting the OS core.
• System Calls: User processes request OS services through system calls, which transition the CPU to kernel mode for execution.
• Auditing: Logging security-relevant events (logins, file access, system changes) for monitoring and forensics.

Cryptography Basics

Cryptography is the science of secure communication using codes and ciphers. It's essential for protecting data confidentiality, integrity, and authenticity in modern systems.

Key Concepts

• Plaintext: The original, readable message.
• Ciphertext: The scrambled, unreadable message after encryption.
• Encryption: The process of converting plaintext to ciphertext.
• Decryption: The process of converting ciphertext back to plaintext.
• Key: A piece of information (a string of bits) used in the encryption and decryption algorithms. The security of the system relies heavily on the secrecy and strength of the key.

Types of Cryptography

1. Symmetric Cryptography (Secret-Key Cryptography)

Uses the same key for both encryption and decryption.

• How it works: Sender and receiver must share a secret key beforehand.
• Algorithms: DES, 3DES, AES (Advanced Encryption Standard), Blowfish.
• Pros: Fast, efficient for large amounts of data.
• Cons: Key distribution problem – securely sharing the secret key between parties is challenging. If the key is compromised, all communication is compromised.

2. Asymmetric Cryptography (Public-Key Cryptography)

Uses a pair of keys: a public key and a private key.

• How it works:
  • Each user generates a key pair (public, private).
  • The public key can be shared freely.
  • The private key must be kept secret.
  • Data encrypted with a public key can only be decrypted with the corresponding private key.
  • Data encrypted with a private key can only be decrypted with the corresponding public key (used for digital signatures).
• Algorithms: RSA, ECC (Elliptic Curve Cryptography), Diffie-Hellman (key exchange).
• Pros: Solves the key distribution problem. Enables digital signatures for authentication and non-repudiation.
• Cons: Much slower and computationally more intensive than symmetric cryptography. Not suitable for encrypting large amounts of data directly.

Hashing

A hash function takes an input message of any size and produces a fixed-size output called a hash value, message digest, or fingerprint.

• Properties:
  • Deterministic: The same input always produces the same output.
  • One-way: It's computationally infeasible to determine the input from the output (preimage resistance).
  • Collision Resistance: It's computationally infeasible to find two different inputs that produce the same output.
• Uses: Data integrity checks (detecting accidental or malicious changes), password storage (store hash of password, not the password itself).
• Algorithms: MD5 (now considered insecure due to collisions), SHA-1 (also deprecated), SHA-256, SHA-3.

Digital Signatures

Provide authentication, integrity, and non-repudiation for digital documents.

• How it works:
  1. The sender creates a hash of the message.
  2. The sender encrypts the hash using their private key. This encrypted hash is the digital signature.
  3. The sender sends the original message along with the digital signature.
  4. The receiver receives the message and the signature.
  5. The receiver creates a hash of the received message.
  6. The receiver decrypts the received digital signature using the sender's public key.
  7. If the hash created by the receiver matches the decrypted hash from the signature, the signature is valid.
• Benefits:
  • Authentication: Confirms the sender's identity (only they have the private key).
  • Integrity: Ensures the message hasn't been altered.
  • Non-repudiation: The sender cannot later deny having sent the message.

Virtualization and Virtual Machines

Virtualization is a technology that allows the creation of virtual versions of computing resources, such as hardware platforms, operating systems, storage devices, or network resources. A Virtual Machine (VM) is the software implementation of a physical computer.

Core Concepts

• Host Machine: The physical computer on which the virtualization software runs.
• Guest Machine: The virtual machine running on the host.
• Hypervisor (Virtual Machine Monitor - VMM): The software layer that creates, runs, and manages virtual machines. It sits between the hardware and the VMs, allocating resources and ensuring isolation.

Types of Hypervisors

1. Type 1 Hypervisor (Bare-Metal)

Runs directly on the host's hardware, without an underlying operating system. It has direct access to hardware resources.

• Examples: VMware ESXi, Microsoft Hyper-V, Xen, KVM (Kernel-based Virtual Machine - integrated into Linux kernel).
• Pros: High performance, efficient resource utilization, strong isolation, more secure as there's no host OS to compromise.
• Cons: Requires dedicated hardware, can be more complex to manage initially.

2. Type 2 Hypervisor (Hosted)

Runs as an application on top of a conventional host operating system (like Windows, macOS, Linux).

• Examples: VMware Workstation, Oracle VirtualBox, Parallels Desktop.
• Pros: Easy to install and use, good for development, testing, and running multiple OSes on a single desktop.
• Cons: Performance overhead because it relies on the host OS for hardware access, less efficient resource utilization, potential security risks if the host OS is compromised.

How Virtualization Works

The hypervisor emulates hardware for each VM. When a guest OS needs to perform an operation (like accessing the disk or network), the hypervisor intercepts these requests.

• CPU Virtualization: The hypervisor manages CPU scheduling for the VMs. Techniques like hardware-assisted virtualization (Intel VT-x, AMD-V) allow VMs to run most instructions directly on the host CPU, improving performance.
• Memory Virtualization: The hypervisor manages physical memory, mapping the virtual memory addresses used by guest OSes to physical addresses on the host.
• I/O Virtualization: The hypervisor intercepts I/O requests from VMs and either emulates devices or uses techniques like paravirtualization or SR-IOV (Single Root I/O Virtualization) for direct hardware access.

Benefits of Virtualization

• Server Consolidation: Run multiple virtual servers on a single physical server, reducing hardware costs, power consumption, and data center space.
• Resource Optimization: Better utilization of hardware resources.
• Isolation: VMs are isolated from each other. A crash or issue in one VM does not affect others or the host.
• Flexibility and Agility: Quickly deploy new servers, clone existing ones, and move VMs between physical hosts (live migration).
• Testing and Development: Create isolated environments for testing software, new OS versions, or risky applications without affecting the production system.
• Disaster Recovery: VMs can be easily backed up, replicated, and restored.
• Legacy Application Support: Run older operating systems and applications on modern hardware.

Virtual Machines (VMs)

A VM is an isolated software container that emulates a complete computer system. It has its own virtual CPU, memory, hard disk, network interface, and other devices.

• Guest OS: You can install any compatible operating system (Windows, Linux, macOS) inside a VM.
• Snapshots: A feature that allows you to save the exact state of a VM at a particular point in time. You can then revert the VM back to this state later, which is incredibly useful for testing or recovery.
• VM Images: The files that represent a VM's virtual hard disk, configuration, and memory state.

Use Cases for Virtual Machines

• Running Multiple Operating Systems: Use Windows on a Mac, or Linux on a Windows PC.
• Software Development and Testing: Developers can test their applications on various OSes and configurations without needing multiple physical machines.
• Sandboxing: Running potentially malicious software or visiting untrusted websites in an isolated VM to protect the host system.
• Cloud Computing: Cloud providers use virtualization extensively to offer virtual servers (e.g., AWS EC2, Google Compute Engine) to customers.
• Education: Students can experiment with different operating systems and software without affecting their main computer.