As a passionate software Engineer you can build your own home server and install Linux on it. I have created this article, sharing some experience doing this work.
Step by Step
Here are the steps to install Linux:
Download the Linux distribution ISO file you want to install. Popular options are Ubuntu, Linux Mint, Fedora, openSUSE etc.
Create a bootable USB drive from the ISO file. You can use software like Rufus, Universal USB Installer, UNetbootin etc.
Boot from the USB drive. You will need to change the boot order in your system's BIOS to boot from the USB drive first.
The Linux installer will launch. Follow the on-screen instructions to install Linux. You will be prompted to:
Choose your language, keyboard layout and location
Partition your hard drive
Create a username and password
Select and install software
Once the installation is complete, restart your system. Linux should now boot from your hard drive instead of the USB drive.
Configure your Linux installation by installing updates, drivers, additional software etc. You can also customize settings as per your preference.
That's the basic step-by-step process to install a Linux distribution on your computer. Let me know if you have any other questions!
Multiple disks
Older PC cases have an internal cage that can support up to 4 or even 6 internal 3.25" disks. You can install multiple internal disks in a RAID array. Here is an example:
How many disks can you actually install depends on your motherboard and power supply PSU. Usually, you can install at least 4 disks, most of the time 6 and very rarely 8 on a home-build server. I have managed to install 10 with an extension card.
If your computer has a 5.25" bay, you can install a special extension to create RAID array even if the computer case is very small and does not have connectors for multiple 3.25" disks. For example, this mount is a good option, but you also need an extension card:
Here are the steps to use multiple hard disks and SSDs to install Linux for better performance and reliability:
- Partition the disks: You will need to partition each disk to use it with Linux. You can create separate partitions for:
/ (root) filesystem: Contains essential files and directories for your Linux installation. Put this on the fastest disk.
/home filesystem: Contains user files and data. Put this on a separate disk for better performance and reliability.
Swap space: Linux swap partition. Can be on any disk.
/var filesystem: Contains log files and data. Put this on a separate disk from / and /home.
Mount the partitions: You need to mount the partitions you created in step 1 during the Linux installation.
Configure fstab: Edit /etc/fstab to make the disk mounts persistent across reboots.
Use RAID: For even more reliability, configure your disks in a RAID array like RAID 1 (mirroring) or RAID 10. This requires a RAID controller or software RAID.
Distribute IO load: By storing different filesystems on separate disks, you distribute the I/O load across multiple disks. This improves performance.
TRIM support: Enable TRIM support in Linux to improve SSD performance and lifespan.
Tune disk scheduler: Tune the I/O scheduler to optimize for low latency (e.g. deadline or CFQ) if using SSDs.
Expansion Card
Most motherboards come with a limited number of SATA ports for connecting internal hard drives and SSDs. SATA expansion cards plug into the motherboard's PCIe slots to provide additional internal SATA ports.
Different motherboards have different PCIe connectors. 1x, 2x, 4x, 8x or 16x. Is possible for you to have only one or two 16x PCIe connectors for the video card and several other connectors for other extension cards?
To install a SATA expansion card:
Determine the correct PCIe slot size required by the card based on the number of internal SATA ports. PCIe x1 slots support 1-2 port cards, while x4 slots support 4-port cards, and so on.
Open the computer case and locate an available PCIe slot of the required size on the motherboard.
Align the card's connector with the slot and firmly press down to secure it.
Attach the SATA cables from your internal drives (HDDs and SSDs) to the connectors on the card.
Install any required drivers or RAID controller software.
Configure the connected drives in your preferred RAID level using the RAID controller or BIOS.
The SATA expansion card will provide additional internal SATA ports, enabling you to connect and utilize more internal drives. These drives can then be organized into a RAID array for increased performance and redundancy.
Special Partitions
When using RAID or multiple disks for Linux installation, there are a few things to keep in mind regarding the /boot and /boot/efi partitions:
- /boot partition:
For BIOS boot (not UEFI), the /boot partition should be on a non-RAID disk. This is because the BIOS can only boot from a single physical disk.
For UEFI boot, the /boot partition can be on a RAID array. This is because UEFI can boot from a logical volume or RAID array.
The /boot partition should be small, around 200-500 MB. It only stores the kernel and initramfs files.
- /boot/efi partition (for UEFI boot only):
This partition stores the UEFI bootloader files.
It should be on the same physical disk as the root (/) partition for simplicity.
It typically needs to be around 300-500 MB in size.
It can be on a RAID array for redundancy.
What file system?
You can use multiple file systems for Linux when using multiple disks and partitions. Different file systems have pros and cons in terms of performance, reliability, features, etc. Some good options are:
/ (root partition) - Ext4: The default Linux file system. Provides good performance, reliability and features for the root filesystem.
/home - Ext4 or Btrfs: Either Ext4 for stability or Btrfs for features like snapshotting and data integrity checking. Depends on your needs.
/var - XFS or Ext4: XFS provides better performance for logging and large files which are common in /var. But Ext4 is more stable.
/tmp - tmpfs: Use a RAM based file system for /tmp for maximum performance. Data in tmpfs is stored in RAM, not on disk.
Swap partition - swap file: Use a swap file instead of a partition. Easier to resize and more flexible.
/boot - Ext2: The Ext2 file system is best suited for the small /boot partition due to its simplicity and reliability.
/boot/efi (UEFI systems) - FAT32: The UEFI specification mandates the usage of FAT32 for the EFI System Partition (ESP).
Web Server
When installing a database and web server, it can be beneficial to install the /srv and /opt directories on separate RAID arrays for better performance:
/srv - Contains site-specific data which is served by the system. This is where your database files would typically live. Placing /srv on a fast RAID array can improve database performance.
A good option would be a RAID 10 array, which provides both performance (striping) and redundancy (mirroring). You want the database files to have fast read/write speeds.
/opt - Contains third-party and optional application software. This is where you would typically install your web server (e.g. Apache or Nginx).
Placing /opt on a separate RAID array from /srv can isolate the performance of the database and web server. A RAID 5 or 6 array could work well here to provide redundancy at a lower cost.
The main benefits would be:
Isolated performance - The database and web server can achieve optimal performance without competing for disk I/O from the same array.
Redundancy - Data is safer with RAID 5 or 10 providing protection against disk failure.
Scalability - You can easily expand the RAID arrays independently as needs grow.
Manageability - Having /srv and /opt on separate arrays makes backup, maintenance and troubleshooting easier.
Boot Loader
The Linux boot loader and EFI work together to start up a Linux system. The boot loader is a small program that loads the Linux kernel and initiates the startup process.
For BIOS systems, the boot loader is GRUB (GRand Unified Bootloader). GRUB loads very early in the boot process and presents a menu to select the Linux kernel to boot.
For UEFI systems, the boot loader is either GRUB or systemd-boot. They are loaded by the UEFI firmware and then load the Linux kernel.
The boot loader is installed in the /boot partition, while configuration files are in /etc/default and /etc/grub.d.
EFI (Extensible Firmware Interface): EFI is a specification that defines a software interface between an operating system and firmware.
UEFI is an implementation of the EFI specification. Modern PCs and laptops generally use UEFI instead of the traditional BIOS.
UEFI provides several benefits over BIOS:
Larger hard drive support (over 2TB)
Faster boot times
Support for loading drivers and operating systems from disk partitions instead of just the MBR
Support for legacy BIOS boot
The EFI System Partition (ESP) is a FAT32 partition that stores the UEFI firmware files, including the Linux boot loader. It is typically mounted at /boot/efi.
So in summary:
The boot loader (GRUB or systemd-boot) is responsible for loading the Linux kernel and initramfs at startup.
EFI/UEFI is the interface between the firmware and operating system that provides features like hard drive support over 2TB and faster boot.
The EFI System Partition stores the boot loader and UEFI firmware files.
Performance with NVMe
M.2 is a new form factor for SSDs that can provide significantly faster performance than traditional SATA SSDs. Here are the main ways M.2 SSDs improve performance over 2.5" SATA SSDs:
Interface - M.2 SSDs use the NVMe interface which has a much higher bandwidth than the SATA interface used by 2.5" SSDs. NVMe can provide up to 32 Gbps of bandwidth while SATA is limited to 6 Gbps.
Connectivity - M.2 SSDs connect directly to the PCIe bus on the motherboard. This provides a more direct connection with lower latency than the indirect connection of SATA through the SATA controller.
Sequential speeds - M.2 SSDs can achieve much higher sequential read/write speeds. While SATA SSDs top out around 550 MB/s, M.2 NVMe SSDs can reach speeds over 3 GB/s.
Random I/O performance - M.2 SSDs also have significantly higher IOPS (input/output operations per second) for random read/write performance. This is important for tasks like booting and application loading.
Lower latency - The direct PCIe connection and higher throughput of M.2 SSDs result in much lower access times (latency). This further improves performance for tasks that involve many small file operations.
Faster boot times - All of these factors combine to allow M.2 NVMe SSDs to significantly reduce boot times and application load times compared to SATA SSDs. Boot time improvements of 30-50% are common.
The combination of the NVMe interface, PCIe connectivity and other design improvements allow M.2 SSDs to provide dramatically higher throughput, IOPS, lower latency and faster boot times compared to traditional 2.5" SATA SSDs. This makes M.2 one of the most meaningful upgrades available for modern motherboards.Faster Setup
For a home server that is used to create a website with database support is better to consider using a combination of SSD + RAID arrays on multiple disks.
Install /srv, which contains the database files, on a fast PCIe NVMe SSD. Using an SSD will provide the best performance for the database, with much faster read/write speeds compared to HDDs.
For even higher performance, consider a RAID 0 array of multiple NVMe SSDs. However, this provides no redundancy so should only be used if data loss is acceptable.
Install /opt, which contains the web server, on a SATA SSD. This will provide a good balance of performance and cost for serving static files and handling web requests.
Use Ext4 file systems on the SSDs for stability. Ext4 has been optimized for SSD performance and endurance.
Install the Linux boot loader to a small SATA SSD for fast boot times.
Leverage UEFI/EFI to take advantage of SSD-optimized features like TRIM support and AHCI mode. This will help maximize the lifespan of the SSDs.
Monitor disk I/O and latency after setup to identify any bottlenecks. Caching, file system tuning, or moving to faster SSDs may help optimize performance further.
Regularly back up the database and web server data to external storage to protect against SSD failure.
In summary, using NVMe and SATA SSDs - isolated for the database and web server - in combination with an optimized Linux configuration, you can achieve extremely fast and reliable performance for your database-driven web application. Let me know if you have any other questions!
Large Files
SSDs are generally better than HDDs for performance due to their faster read/write speeds. However, for large /var files like logs, caches and temporary files, RAID arrays can be a better option:
Endurance: SSDs have a limited number of write cycles before cells start to fail. Large and constantly changing /var files can wear out SSD cells quickly. RAID arrays using HDDs have much higher endurance for sustained write workloads.
Over-provisioning: SSD controllers reserve a portion of storage as over-provisioning to help with wear leveling and performance. Large /var files can fill up an SSD's over-provisioned space, impacting performance and lifespan. HDD RAID arrays do not have this issue.
Latency: While SSD read speeds are faster, write speeds can be slower for large files due to the flash translation layer (FTL) on the SSD controller. HDD RAID arrays often have more consistent latency for writing large files.
Cost: HDD RAID arrays tend to offer lower $/GB compared to SSDs, especially for higher capacity needs. The cost savings can be significant for /var partitions that grow over time.
Reliability: RAID arrays provide redundancy which is important for /var files. A single SSD failure could result in data loss. RAID 5/6/10 arrays using HDDs offer data protection.
In summary, while SSDs are generally faster, HDD RAID arrays are often a better fit for /var partitions due to their higher endurance for sustained writes, more consistent latency for large files, lower cost per gigabyte, and built-in redundancy. This makes HDD RAID a more reliable and cost-effective choice for logs, caches and temporary files.
SAS Disks
SATA hard drives are designed for consumer-grade single-drive computers while SAS hard drives are designed for enterprise servers and RAID arrays. This leads to some key differences in reliability:
Data transfer speed: SAS drives have faster data transfer speeds of up to 6Gbps compared to SATA drives which are limited to 3Gbps. The higher throughput of SAS drives makes them better suited for RAID.
Error correction: SAS drives have an Advanced Error Correction Code (ECC) which can detect and correct more types of read errors compared to SATA drives. This improves data integrity in RAID arrays where drive failures are more common.
Drive communication: SAS drives have a point-to-point connection to the controller while SATA drives use a shared bus architecture. The direct connections in SAS are more reliable for multipath RAID configurations.
Drive firmware: SAS drives have enterprise-grade firmware designed for running 24/7 in server environments. The firmware is more robust and stable compared to consumer-focused SATA drive firmware.
Mean time between failures (MTBF): SAS drives are engineered with better components and higher quality standards, resulting in MTBF ratings that are 2-4x higher than SATA drives. This makes SAS drives more dependable for mission-critical applications.
Drive monitoring: SAS drives support more health monitoring features like S.M.A.R.T which can detect reliability issues proactively. This reduces unplanned downtime in RAID arrays.
Warning: SAS disks require a SAS controller or RAID card to be installed and used properly in a PC. Here are the reasons why:
Connectivity - SAS disks have a different connector than SATA disks. They use a narrower SAS connector, not the wider SATA connector. Therefore, a standard SATA controller on the motherboard cannot connect to SAS disks.
Signaling - SAS disks use a different signaling standard called SAS than SATA disks, which use SATA. A standard SATA controller cannot communicate with SAS disks using the SAS signaling protocol.
Drive compatibility - SAS controllers are designed to work specifically with SAS disks. They have the firmware and drivers to support the features and capabilities of SAS disks.
Hardware RAID - If you want to configure SAS disks in a RAID array for things like redundancy, performance or capacity, you need an SAS RAID card or controller. Standard SATA controllers do not support RAID arrays with SAS disks.
Performance - SAS controllers and RAID cards can provide the full throughput that SAS disks are capable of. A standard SATA controller would bottleneck the performance of SAS disks.
So to properly install and use SAS disks in a PC, you need an additional PCIe expansion card that has either a SAS controller or a SAS RAID controller. These cards will provide the correct connectors, signaling, firmware, drivers and hardware RAID support that SAS disks require.
Clarification: Enterprise servers have a different kind of motherboard that has support for SAS disks native on the motherboard and does not use SATA interface. Due to the higher cost, and compatibility issues, using SAS disks is not a good upgrade for my home server.
Choosing RAID Level
Using SATA disks for RAID5 or RAID6 can cause performance and reliability issues. It is recommended to use RAID1 or RAID10 for SATA disks, and RAID5/RAID6 for SAS enterprise disks.
Reasons to avoid RAID5/RAID6 with SATA disks:
SATA disks have slower read/write speeds and cannot fully utilize the throughput of RAID5 and RAID6 arrays. This can lead to poor performance.
The additional parity calculations in RAID5/RAID6 put more stress on the disk controllers of consumer-grade SATA disks. This can impact reliability over time.
Some lower-cost "green" SATA disks are designed for low power, not high performance. They may not be able to handle the workload of RAID5/RAID6 arrays.
SATA disks have lower MTBF ratings, so multiple simultaneous failures are more likely. RAID5/RAID6 does not protect against more than one failure.
Advantages of RAID1/RAID10 for SATA disks:
Mirroring (RAID1) and stripe-and-mirror (RAID10) require no parity calculations, placing less stress on SATA disk controllers.
By doubling the number of disks, RAID1 and RAID10 can protect against multiple simultaneous drive failures.
The read performance of RAID1 and RAID10 will match the read speed of the fastest SATA disk, providing adequate performance for most use cases.
Older Motherboards
Some older motherboards offer SATA2 and SATA3 connectors. Here are the key points regarding using SATA2 vs SATA3 on an older motherboard with both types of connectors:
SATA3 (also known as SATA 6Gbps) has double the bandwidth of SATA2 (3Gbps). This means it can achieve much higher read/write speeds, up to 600 MB/s. SATA2 is limited to around 300 MB/s.
For SSDs, using a SATA3 port will allow the drive to achieve closer to its maximum rated performance. So on this motherboard, it's recommended to connect any SSDs to the SATA3 ports to get the best speeds.
For HDDs in a RAID array, the difference between SATA2 and SATA3 is less significant since HDDs have lower maximum speeds to begin with. Either port type will likely saturate the HDD's bandwidth.
The two SATA3 ports on this motherboard are likely to be wired to the most modern part of the chipset, so they may have slightly better performance and reliability.
For both SSDs and HDD RAID arrays, using the SATA3 ports will give you more future-proofing in case you upgrade to faster drives in the future that can take advantage of the extra bandwidth. Avoid SATA2 if you can afford a new motherboard.
In summary, on this older motherboard, I would recommend:
Connect any SSDs to the 2 available SATA3 ports to get the full performance benefits of the SSD.
Connect the HDD RAID array to any of the 4 SATA2 ports. The difference in speed will be minimal and either port will likely saturate the HDDs' bandwidth.
Consider using the SATA3 ports for the RAID array as well to give more headroom for future drive upgrades, though the performance gain may not be significant with today's HDDs.
Hope this will help you to optimize drive connections on your motherboard and avoid mistakes I have made. Installing SSD on the wrong SATA ports, and losing significant performance of SSD.
Conclusion: After studying these things, I have installed Linux on 10 disks. I have used 6 SATA provided by the motherboard and an additional 4 SATA provided using a RAID extension card. I have used only RAID1 and RAID10 to avoid performance issues with SATA disks and I have used SATA3 for SSD.