Managing and Troubleshooting Linux Performance

Hi everyone! In this article, I will help you in troubleshooting the Linux virtual machine performance.

Troubleshooting and Managing CPU Load and Memory Issues

proc and sys directory manage hardware resources

ls /
cd /proc
cat cpuinfo
less cpuinfo | grep processor

CPU Capacity Limits

If the number is at 75% or above capacity then the application is close to demanding more resources. To deal with that:

Reduce load
Add hardware capacity

Using Commands to manage CPU and Memory Loads:

uptime

# Four Processor
# 1.0 = 25% capacity
# 2.0 = 50% capacity

top
# press `c` in the top command prompt to get the full path of the command executing
# press `d` to change the refresh time of the top command (i.e delay time)
# press `k` to kill the process
# press `Shift + p` to sort the process by CPU utilization
# press`Shift + h` to display threads
# press `Shift + v` to get child-parent structure
# press 1 to get total CPUs
# press `u` to get specific user processes
# press `h` for help

journalctl _PID=<process_id>

Memory Usage

Physical Memory (RAM)
SWAP (Allocated disk space)
Disk Caching (RAM borrowed for application use)

cat /proc/meminfo
free -h
vmstat
    # vmstat display 
    r: Total number of processes waiting for CPU time
    b: Total number of blocked processes, waiting for disk or network IO
    swpd: Used virtual memory
    free: Free virtual memory
    buff: Memory used as a buffer (whats in directories, permission)
    cache: Memory used as Cache
    si: Memory swapped from disk every second
    so: Memory swapped to disk every second
    bi: Blocks in per second
    bo: Blocks out per second
    in: Interrupts per second
    cs: Context switches per second

# memory usage in MB
vmstat -S M

# Display used/inused memory
vmstat -a

# Running with delay and repeated times
vmstat 5 3    # run 3 times with 5 seconds delay

## Free up memory space Immediately
free -m
sync
sudo bash -c "echo 3 > /proc/sys/vm/drop_caches"
free -m

Managing CPU and Memory Usage with cgroups

# create a group: testgroup, -a: username
cgcreate -a izhar -g cpu:testgroup
cd /sys/fs/cgroup
cd cpu
cat cpu.cfs_period_us

# running command
cgexec -g cpu:testgroup <cmd/program>

cd /sys/fs/cgroup/memory
ls
cat memory.limit_in_bytes
cgcreate -a izhar -g memory:newgroup

# set memory limit
cgset -r memory.limit_in_bytes=512m newgroup
cgexec -g memory:newgroup <cmd/program>

Storage and Network Issues

# controlling logs
ls /var/log

# setting for logs
cat /etc/logrotate.conf

# default setting for logs
cat /etc/logrotate.d/ufw

# journald logs
ls /var/log/journal/
less /etc/systemd/journald.conf

Working with Inode

    +-----------+-------+
    | File Name | Inode |
    +-----------+---|---+
                    |
      +-------------|------+-----+-----+--------------+----------------+
      | Last Access | Size | UID | GID | Direct Links | Indirect Links |
      +--------------------+-----+-----+-----------|--+----------------+
        +------------+------------+------------+---|        
        | Data Block | Data Block | Data Block | 
        +------------+------------+------------+


df -ih

# finding files with highest number of inodes
find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort -n

Network Bandwith

iftop -i <networkcard>
    # in new terminal run commands like curl https://google.com to monitor traffic
nethogs <networkcard>

Thank you for Reading!