Process Management: Knowing What is Running on Your System – Part 4 of 5 of the Terminal Techniques for You (TTY): Making Linux Security Accessible Blog Series

Welcome to the fourth installment in our TTY series! So far, we've explored how to navigate the Linux file system, keep your software updated, and control file permissions. In this post, we're going to discover another critical aspect of Linux security: understanding and managing the processes running on your system.

Every program that runs on your Linux system, from the graphical user interface and background services that manage your network connections, exists as a process. Knowing how to monitor, control, and secure these processes is essential for maintaining a healthy and secure system.

What Is a Process?

A process is simply a running instance of a program. When you launch an application, the operating system loads the program's code from disk into memory and begins executing it. Each process has its own:

• Process ID (PID): A unique number that identifies the process
• Owner: The user who started the process
• Resource Allocations: Memory, CPU time, and other resources
• Security Context: Permissions that determine what the process can do

Understanding processes gives you visibility into what's happening on your system and helps you identify potential security issues.

Viewing Running Processes

Let's start with the basic commands for viewing processes.

#### The ps Command: Process Snapshot

The `ps` command provides a snapshot of current processes:

$ ps

PID TTY          TIME CMD

2325 pts/0    00:00:00 bash

3438 pts/0    00:00:00 ps

This basic output shows only processes associated with your current terminal session. For a more detailed and expansive view, we need to add options:

$ ps aux

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

root           1  0.0  0.1 169236 13216 ?        Ss   Mar01   0:02 /sbin/init

root         437  0.0  0.1  72700  9856 ?        Ss   Mar01   0:00 /usr/sbin/sshd -D

sec406       1035  0.0  0.2 573996 23736 ?        Ssl  Mar01   0:01 /usr/bin/gnome-session

sec406       2325  0.0  0.0  21452  5636 pts/0    Ss   10:15   0:00 bash

sec406       3442  0.0  0.0  39132  3604 pts/0    R+   10:30   0:00 ps aux

Here's what the options mean:

• `a`: Show processes from all users
• `u`: Show detailed information including the owner
• `x`: Include processes not attached to a terminal

The output columns tell us:

• USER: Who owns the process
• PID: Process ID number
• %CPU: Percentage of CPU being used
• %MEM: Percentage of RAM being used
• VSZ: Virtual memory size
• RSS: Actual physical memory used
• TTY: Terminal associated with the process (? means no terminal)
• STAT: Process status (R=running, S=sleeping, etc.)
• START: When the process started
• TIME: CPU time used
• COMMAND: The command that started the process

For security purposes, pay attention to:

1. The USER column (who's running what)
2. Any processes with high resource usage (CPU or memory)
3. Unfamiliar process names, which may indicate unauthorized activity

#### The top Command: Real-time Process Monitoring

While `ps` gives us a snapshot in time of processes on the system, `top` provides a "real-time," continuously updating view:

$ top

This shows a similar display to `ps aux`, but updates every few seconds (by default), highlighting processes that are currently active. Press 'q' to exit.

Useful keys within top:

• `k`: Kill a process (you'll be prompted for the PID)
• `r`: Renice a process (change its priority)
• `f`: Select different fields to display
• `u`: Filter by user
• `M`: Sort by memory usage
• `P`: Sort by CPU usage (default)

#### The htop Command: A User-Friendly Alternative

If available on your system, `htop` provides what some consider a more user-friendly interface with color coding and easier navigation.

$ htop

If you do not have it installed, you can install it and check it out:

$ sudo apt install htop# For Debian/Ubuntu

Not sure what that `apt` command you just ran did? Refer to the the second post in this series."

Understanding Process Ownership and Privileges

Every process runs under the user ID of a specific user account, known as the effective user ID (EUID). This is a fundamental security concept in Linux:

• Processes inherit the permissions of their owner
• Root-owned processes have full system access
• Regular user processes have limited access based on the user's permissions

To see which processes belong to a specific user:

$ ps -u sec406

This process ownership is why maintaining separate user accounts and following the principle of least privilege is important for security.

Process Relationships: Parents and Children

Processes in Linux have parent-child relationships. When a process starts another process, the original one becomes the parent and the new one is the child.

To see these relationships:

$ ps f

PID TTY      STAT   TIME COMMAND

2325 pts/0    Ss     0:00 bash

3456 pts/0    R+     0:00  \_ ps f

The `f` option shows a "forest" view with lines connecting related processes. This is useful to understand how processes started and what might be affected if you terminate a parent process.

To see a more detailed view of process relationships, use the `pstree` command:

$ pstree

systemd─┬─ModemManager───2*[{ModemManager}]

├─NetworkManager───2*[{NetworkManager}]

├─accounts-daemon───2*[{accounts-daemon}]

├─acpid

├─avahi-daemon───avahi-daemon

├─bluetoothd

├─cron

├─cups-browsed───2*[{cups-browsed}]

├─cupsd

├─dbus-daemon

├─gdm3─┬─gdm-session-wor─┬─gdm-x-session─┬─Xorg

│      │                 │               ├─gnome-session-b─┬─ssh-agent

│      │                 │               │                 └─2*[{gnome-session-b}]

│      │                 │               └─2*[{gdm-x-session}]

│      │                 └─2*[{gdm-session-wor}]

│      └─2*[{gdm3}]

├─networkd-dispat

├─polkitd───2*[{polkitd}]

├─rsyslogd───3*[{rsyslogd}]

├─rtkit-daemon───2*[{rtkit-daemon}]

├─snapd───10*[{snapd}]

├─systemd─┬─(sd-pam)

│         ├─at-spi-bus-laun─┬─dbus-daemon

│         │                 └─3*[{at-spi-bus-laun}]

│         ├─at-spi2-registr───2*[{at-spi2-registr}]

│         ├─dbus-daemon

│         ├─dconf-service───2*[{dconf-service}]

│         ├─evolution-addre───5*[{evolution-addre}]

│         ├─evolution-calen───8*[{evolution-calen}]

│         ├─evolution-sourc───3*[{evolution-sourc}]

│         ├─gnome-shell─┬─ibus-daemon─┬─ibus-engine-sim───2*[{ibus-engine-sim}]

│         │             │             └─2*[{ibus-daemon}]

│         │             └─14*[{gnome-shell}]

│         ├─gnome-terminal-─┬─bash───pstree

│         │                 └─3*[{gnome-terminal-}]

This shows the complete hierarchy of processes, with each indentation level representing a parent-child relationship.

Process States: What's It Doing?

The STAT column in `ps` output shows the current state of each process:

• `R`: Running or runnable (in the queue to run)
• `S`: Interruptible sleep (waiting for an event)
• `D`: Uninterruptible sleep (usually waiting for I/O)
• `Z`: Zombie (terminated but not yet cleaned up)
• `T`: Stopped (suspended)

Additional characters may appear after these:

• `+`: In the foreground process group
• `<`: High-priority process
• `N`: Low-priority process
• `s`: Session leader

For security monitoring, pay attention to:

• Zombie processes (`Z`), which might indicate software bugs
• High-priority processes (`<`), which might attempt to monopolize/hog system resources
• Processes in uninterruptible sleep (`D`) for extended periods, which might be stuck

Controlling Processes

Now that we can see processes, let's learn how to control them!

#### Starting Processes

There are several ways to start a process:

1. Run a command in the foreground:

$ firefox

This occupies your terminal until the program exits. Launching a process in this manner makes your shell prompt disappear and you will not be able to issue commands to the terminal until the process exits or is running in the background.

2. Run a command in the background with `&`:

$ firefox &

[1] 3531

The number in brackets is the job number (or jobspec) and the number after it is the PID. It may seem like you have lost your terminal prompt when you run a command with an `&` after it, but often you just need to hit ENTER and your prompt returns. However, if you close your terminal/shell session, that process will probably terminate/die.

3. Start a program and detach it completely:

$ nohup firefox &

The `nohup` command makes the process immune to hangup signals, so it continues running even if you log out.

#### Stopping Processes

To stop a running process, you can use the `kill` command with the process ID:

$ kill 3531

This sends a polite termination signal (SIGTERM) to the process, asking it to shut down gracefully. If that doesn't work, you can force it:

$ kill -9 3531

The `-9` option sends the SIGKILL process signal, which forcibly terminates the process. This should be used as a last resort because it doesn't allow the program to clean up.

For processes running in your terminal, you can often use keyboard shortcuts:

• CTRL+C: Sends an interrupt signal (SIGINT), similar to SIGTERM
• CTRL+Z: Suspends the process (puts it in the stopped state)

#### Managing Background Jobs

When you run commands with `&` or suspend them with CTRL+Z, they become background jobs. You can manage backgrounded jobs with:

$ jobs

$ fg %1

$ bg %1

The `%1` refers to the job number, or jobspec, shown in the `jobs` output.

Monitoring System Resources

Understanding resource usage helps identify abnormal behavior that might indicate security issues.

#### Memory Usage

The `free` command shows memory usage:

$ free -h

total        used        free      shared  buff/cache   available

Mem:          7.7Gi       2.1Gi       3.7Gi       279Mi       1.9Gi       5.1Gi

Swap:         2.0Gi          0B       2.0Gi

The `-h` option shows values in human-readable format (gigabytes/megabytes), otherwise you will see bytes.

#### Disk Usage

The `df` command shows disk space usage:

$ df -h

Filesystem      Size  Used Avail Use% Mounted on

/dev/sda1       234G   32G  191G  15% /

tmpfs           3.9G     0  3.9G   0% /dev/shm

/dev/sda2       511M  7.8M  504M   2% /boot/efi

To see disk usage for a specific directory:

$ du -sh /home/sec406

4.2G    /home/sec406

#### Network Connections

The `netstat` or `ss` commands show network connections:

$ ss -tuln

Netid  State   Recv-Q  Send-Q     Local Address:Port     Peer Address:Port

udp    UNCONN  0       0          127.0.0.53%lo:53       0.0.0.0:*

udp    UNCONN  0       0          0.0.0.0:68             0.0.0.0:*

tcp    LISTEN  0       128        127.0.0.53%lo:53       0.0.0.0:*

tcp    LISTEN  0       128        0.0.0.0:22             0.0.0.0:*

tcp    LISTEN  0       5          127.0.0.1:631          0.0.0.0:*

tcp    LISTEN  0       128        [::]:22                [::]:*

tcp    LISTEN  0       5          [::1]:631              [::]:*

The options mean:

• `t`: TCP connections
• `u`: UDP connections
• `l`: Listening sockets
• `n`: Show numerical addresses instead of resolving names

This helps identify which services are accessible over the network. From a security perspective, every open port is a potential entry point that needs to be monitored and protected with a defensive measure.

Finding Which Process Is Using a Port

If you see an open port and want to know which process is using it:

$ sudo lsof -i :22

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

sshd     437 root    3u  IPv4  17109      0t0  TCP *:ssh (LISTEN)

sshd     437 root    4u  IPv6  17111      0t0  TCP *:ssh (LISTEN)

This shows that the `sshd` process (PID 437) is listening on port 22. The `lsof` utility is not always installed by default, so you may need to install it before use.

Identifying Resource-Intensive Processes

To find processes using a lot of system resources:

$ ps aux --sort=-%cpu | head -10

$ ps aux --sort=-%mem | head -10

Unexpected resource usage can be a sign of malware, compromise, or a system problem affecting availability.

Understanding Services

Many processes run as services; background programs that start automatically and provide system functionality. In most modern Linux distributions, services are managed by `systemd`.

To list all active services:

$ systemctl list-units --type=service

To check the status of a specific service:

$ systemctl status ssh

● ssh.service - OpenBSD Secure Shell server

Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)

Active: active (running) since Tue 2023-03-01 09:20:14 UTC; 4 days ago

Docs: man:sshd(8)

man:sshd_config(5)

Main PID: 437 (sshd)

Tasks: 1 (limit: 4666)

Memory: 9.9M

CGroup: /system.slice/ssh.service

└─437 /usr/sbin/sshd -D

This shows the service status, when it started, its documentation, main process ID, and resource usage.

#### Starting and Stopping Services

To control services:

$ sudo systemctl start apache2

$ sudo systemctl stop apache2

$ sudo systemctl restart apache2

$ sudo systemctl enable apache2

$ sudo systemctl disable apache2

From a security perspective, disable any services you don't need to reduce a system's attack surface.

Security Implications of Processes

Now that we understand the basics of processes, let's focus on their security implications.

#### Principle of Least Privilege for Processes

Just like users, processes should only have the permissions they need to function. This means:

1. Avoid running services as root when possible
2. Use dedicated service users with limited permissions
3. Apply appropriate file permissions to limit what processes can access

Many modern services automatically drop privileges after starting:

$ ps aux | grep apache2

root       873  0.0  0.1  74064  5616 ?        Ss   Mar01   0:01 /usr/sbin/apache2 -k start

www-data   875  0.0  0.1 126668  6564 ?        S    Mar01   0:00 /usr/sbin/apache2 -k start

www-data   876  0.0  0.1 126668  6564 ?        S    Mar01   0:00 /usr/sbin/apache2 -k start

In the example above, Apache starts as root (to bind to privileged ports) but spawns other worker processes as the less-privileged `www-data` user.

#### Identifying Suspicious Processes

Several characteristics might indicate a suspicious process:

1. Unusual names or a misspelling of a common programs
2. Processes running from unexpected locations
3. Unexpected network connections
4. High resource usage with no apparent reason
5. Processes running as an unexpected user

To investigate a suspicious process:

$ ps -p 1234 -f

$ sudo lsof -p 1234

$ sudo lsof -i -a -p 1234

$ sudo strace -p 1234

Limiting Process Resources

To prevent processes from consuming excessive resources, you can use various tools.

#### Setting Resource Limits with ulimit

The `ulimit` command sets limits for the current shell and its child processes:

$ ulimit -a

$ ulimit -f 1000# 1000 blocks (512KB)

$ ulimit -u 50

To make these settings permanent, they can be added to configuration files like `/etc/security/limits.conf`.

#### Using cgroups for Process Control

Modern Linux systems use control groups (cgroups) for more fine-grained resource management. With `systemd`, you can set limits for services:

$ sudo systemctl set-property apache2.service CPUQuota=20%

$ sudo systemctl set-property apache2.service MemoryLimit=512M

This limits the Apache service to 20% of CPU and 512MB of memory.

Monitoring Tools

For comprehensive monitoring, consider tools that provide advanced detection and alerting capabilities like:

• `sysstat` for performance monitoring
• `monit` for process monitoring and automatic restart
• `prometheus` with `node_exporter` for metrics collection
• `auditd` for detailed system auditing

Practical Exercise: Process Exploration

Let's apply what we've learned with a practical exploration:

1. Check what's currently running:

$ ps aux

2. Identify the top resource users:

$ top

Press 'q' to exit.

3. Look for network connections:

$ sudo ss -tuln

4. Find which processes are making network connections:

$ sudo lsof -i

5. Check for any unusual processes running as root:

$ ps -U root -u root u

6. Monitor file system activity:

$ sudo apt install iotop# Install if needed

$ sudo iotop

Press 'q' to exit.

7. Create a simple script to monitor CPU usage and save it as ~/scripts/monitor.sh:

#!/bin/bash

while true; do

date

ps -eo pid,ppid,cmd,%cpu,%mem --sort=-%cpu | head -10

echo "-----------------------------------"

sleep 5

done

Make it executable and run it:

$ chmod u+x ~/scripts/monitor.sh

$ ~/scripts/monitor.sh

Press CTRL+C to stop the script.

Process Isolation with Containers

For enhanced security, especially for network services, consider using a container runtime like Docker or LXC that can provide process isolation. Follow the instructions to install Docker or LXC on your distribution, then:

$ sudo docker run -d -p 8080:80 nginx

This runs the Nginx web server in an isolated environment with limited access to your main system, reducing the potential impact of a compromise.

Conclusion: Processes and Security Awareness

Understanding and managing processes is a crucial aspect of Linux security. By monitoring what's running on your system, you can:

1. Identify unauthorized or suspicious activity
2. Control what services are exposed to potential attacks
3. Limit resource usage to prevent denial of service
4. Ensure processes have only the permissions they need

This knowledge builds on the file system navigation, package management, and permission controls we've covered in previous posts. By combining these skills, you're developing a comprehensive approach to Linux security.

Practice Questions

To reinforce your learning, try answering these questions:

1. What command shows all processes owned by the user named "webserver"?
2. A process is using 100% CPU. What command do you use to reduce its priority without killing it?
3. How do you check which process is listening on port 80?
4. How do you run a long process that persists after you log out? What command prefix would you use?
5. What's the difference between `kill` and `kill -9`, and when should you use each?

Want to know more?

Check out the course preview of SEC406: Linux Security for InfoSec Professionals for a free hour of course content. Ready to take your Linux skills to the next level? For a limited time, take SEC406 for just $5,250!

Coming up next on TTY, we'll explore network security basics, building on this process knowledge to understand how your system interacts with the network and how to secure those interactions.