Installing and configuring sysstat on CentOS 8

Sysstat is a package of performance monitoring tools for Linux operating systems. It provides a collection of utilities to collect, process and analyze system utilization data over time. The sysstat package includes tools like sar, iostat, mpstat, and pidstat, which provide various system performance metrics such as CPU, memory, disk and network usage, process statistics, and more. The data collected by sysstat can be used to identify performance bottlenecks, troubleshoot system issues, and make informed decisions about system resource usage.

This article shows how to install sysstat and to check the system.

Overview

The SYSSTAT software application is composed of several utilities. Each utility has a specific function:

  • iostat reports CPU statistics and input/output statistics for devices, partitions and network filesystems.
  • mpstat reports individual or combined processor related statistics.
  • pidstat reports statistics for Linux tasks (processes) : I/O, CPU, memory, etc.
  • sar collects, reports and saves system activity information (CPU, memory, disks, interrupts, network interfaces, TTY, kernel tables, NFS, sockets etc.)
  • sadc is the system activity data collector, used as a backend for sar.
  • sa1 collects and stores binary data in the system activity daily data file. It is a front end to sadc designed to be run from cron.
  • sa2 writes a summarized daily activity report. It is a front end to sar designed to be run from cron.
  • sadf displays data collected by sar in multiple formats (CSV, XML, etc.) This is useful to load performance data into a database, or import them in a spreadsheet to make graphs.

Let’s start by installing Sysstat

yum -y install sysstat

Enable Sysstat

systemctl enable sysstat

Check the cron job

cat /etc/cron.d/sysstat

Output

cat /etc/cron.d/sysstat

Output:

# The first element of the path is a directory where the debian-sa1
# script is located
PATH=/usr/lib/sysstat:/usr/sbin:/usr/sbin:/usr/bin:/sbin:/bin

# Activity reports every 10 minutes everyday
5-55/10 * * * * root command -v debian-sa1 > /dev/null && debian-sa1 1 1

# Additional run at 23:59 to rotate the statistics file
59 23 * * * root command -v debian-sa1 > /dev/null && debian-sa1 60 2

Enable SADC

nano /etc/default/sysstat

Output

Change ENABLED=”false” to ENABLED=”true”. Then start sysstat.

#
# Default settings for /etc/init.d/sysstat, /etc/cron.d/sysstat
# and /etc/cron.daily/sysstat files
#

# Should sadc collect system activity informations? Valid values
# are "true" and "false". Please do not put other values, they
# will be overwritten by debconf!
ENABLED="true"
systemctl restart sysstat

Usage Examples

NOTE: Sysstat takes time to gather data. If you see no real output from the below commands, give the system time to generate data.

View RAM usage

sar -r

Output

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

06:58:35 PM  LINUX RESTART      (1 CPU)

07:00:09 PM kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
07:10:09 PM   1277964   1627632    240072     11.82     23992    438200   1097844     36.24    161488    493468       132
07:20:01 PM   1277696   1627448    240244     11.83     24072    438204   1098864     36.28    161572    494000       156
07:30:09 PM   1277428   1627272    240424     11.84     24160    438204   1098016     36.25    161656    493696       176
Average:      1277696   1627451    240247     11.83     24075    438203   1098241     36.26    161572    493721       155

View Swap usage

sar -S

Output

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

06:58:35 PM  LINUX RESTART      (1 CPU)

07:00:09 PM kbswpfree kbswpused  %swpused  kbswpcad   %swpcad
07:10:09 PM    998396         0      0.00         0      0.00
07:20:01 PM    998396         0      0.00         0      0.00
07:30:09 PM    998396         0      0.00         0      0.00
Average:       998396         0      0.00         0      0.00

It’s possible to output current or past status of system with “sar” command like follows. Output the past CPU usage from the log file.

Find the current file:

ls -l /var/log/sysstat/
total 8
-rw-r--r-- 1 root root 4792 Mar 13 19:10 sa13

The log file is “sa13.” Run the following:

sar -u -f /var/log/sysstat/sa13

Output:

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

06:58:35 PM  LINUX RESTART      (1 CPU)

07:00:09 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
07:10:09 PM     all      0.03      0.00      0.02      0.03      0.01     99.91
07:20:01 PM     all      0.02      0.00      0.02      0.02      0.02     99.93
07:30:09 PM     all      0.02      0.00      0.02      0.02      0.01     99.93
Average:        all      0.02      0.00      0.02      0.02      0.01     99.92

Output the current CPU usage for 3 times every second.

sar -u 1 3

Output

Linux 2.6.32-358.6.2.el6.x86_64 (dlp.server.world)      06/24/2013      _x86_64_        (2 CPU)

05:29:12 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
05:29:13 PM     all      0.00      0.00      0.00      0.00      0.00    100.00
05:29:14 PM     all      0.00      0.00      0.00      0.00      0.50     99.50
05:29:15 PM     all      0.50      0.00      0.50      0.00      0.00     99.00
Average:        all      0.17      0.00      0.17      0.00      0.17     99.50

Output the current Disk usage for 3 times every second.

sar -b 1 3

Output:

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

07:37:20 PM       tps      rtps      wtps      dtps   bread/s   bwrtn/s   bdscd/s
07:37:21 PM      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:37:22 PM      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:37:23 PM      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:         0.00      0.00      0.00      0.00      0.00      0.00      0.00

Output the current Memory usage for 3 times every second.

sar -r 1 3

Output

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

07:37:45 PM kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
07:37:46 PM   1277404   1627348    240340     11.84     24224    438232   1098380     36.26    161744    493800         0
07:37:47 PM   1277404   1627348    240340     11.84     24224    438232   1098380     36.26    161744    493800         0
07:37:48 PM   1277404   1627348    240340     11.84     24224    438232   1098380     36.26    161744    493800         0
Average:      1277404   1627348    240340     11.84     24224    438232   1098380     36.26    161744    493800         0

Output the current send/receive packets for 3 times every second.

sar -n DEV 1 3

Output:

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

07:19:58 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
07:19:59 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:19:59 PM    enp1s0      2.00      0.00      0.11      0.00      0.00      0.00      0.00      0.00

07:19:59 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
07:20:00 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:20:00 PM    enp1s0      1.00      1.00      0.06      0.62      0.00      0.00      0.00      0.00

07:20:00 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
07:20:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
07:20:01 PM    enp1s0      2.00      1.00      0.11      0.62      0.00      0.00      0.00      0.00

Average:        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
Average:           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:       enp1s0      1.67      0.67      0.09      0.41      0.00      0.00      0.00      0.00

Output the current paging usage for 3 times every second.

sar -B 1 3

Output

07:22:09 PM  pgpgin/s pgpgout/s   fault/s  majflt/s  pgfree/s pgscank/s pgscand/s pgsteal/s    %vmeff
07:22:10 PM      0.00      0.00      0.00      0.00     33.00      0.00      0.00      0.00      0.00
07:22:11 PM      0.00      0.00      0.00      0.00      1.00      0.00      0.00      0.00      0.00
07:22:12 PM      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:         0.00      0.00      0.00      0.00     11.33      0.00      0.00      0.00      0.00

It’s possible to output every CPU cores’ usage with “mpstat” command like follows. Output the current all CPU cores’ usage for 3 times every second.

mpstat -P ALL 1 3

Output

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

07:24:09 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
07:24:10 PM  all    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
07:24:10 PM    0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00

07:24:10 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
07:24:11 PM  all    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
07:24:11 PM    0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00

07:24:11 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
07:24:12 PM  all    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
07:24:12 PM    0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00

Average:     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
Average:     all    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00
Average:       0    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00

It’s possible to output disk I/O usage with “iostat” command like follows. Output the current I/O usage with Mega-bytes for 3 times every second.

iostat -mx 1 3

Output

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.07    0.00    0.09    0.23    0.01   99.60

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
loop0            0.00      0.00     0.00   0.00   55.08    21.62    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.01
loop1            0.05      0.00     0.00   0.00    6.78    29.62    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.04
loop2            0.00      0.00     0.00   0.00   28.70     7.24    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.01
loop3            0.00      0.00     0.00   0.00   31.74     8.05    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.01
loop4            0.00      0.00     0.00   0.00    0.00     1.50    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
sr0              0.00      0.00     0.00   0.00    0.11     0.17    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
vda              0.60      0.03     0.23  27.39   15.19    46.77    0.32      0.01     0.32  49.34    5.36    19.55    0.00      0.00     0.00   0.00    0.00     0.00    0.05   17.73    0.01   0.30


avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.00    0.00    1.00    0.00    0.00   98.00

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
loop0            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
loop1            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
loop2            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
loop3            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
loop4            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
sr0              0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
vda              0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00


avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    0.00    0.00    0.00  100.00

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
loop0            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
loop1            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
loop2            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
loop3            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
loop4            0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
sr0              0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00
vda              0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00

It’s possible to output every CPU usage for processes with “pidstat” command like follows. Output the CPU usage for a process ID “1169” for 3 times every second.

pidstat -p 1169 1 3

Output

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

04:09:09 PM       PID    %usr %system  %guest    %CPU   CPU  Command
04:09:10 PM      1169    0.00    0.00    0.00    0.00     1  bash
04:09:11 PM      1169    0.00    0.00    0.00    0.00     1  bash
04:09:12 PM      1169    0.00    0.00    0.00    0.00     1  bash
Average:         1169    0.00    0.00    0.00    0.00     -  bash

Viewing current processes

pidstat -d

Output

Linux 5.10.0-21-amd64 (dns.ipgw.io)     03/13/2023      _x86_64_        (1 CPU)

08:07:05 PM   UID       PID   kB_rd/s   kB_wr/s kB_ccwr/s iodelay  Command
08:07:05 PM     0         1      6.38      0.01      0.00     375  systemd
08:07:05 PM     0       141      0.00      0.00      0.00       1  kworker/0:3-events
08:07:05 PM     0       173      0.00      0.52      0.00    1466  jbd2/vda1-8
08:07:05 PM     0       212      0.46      0.97      0.00     403  systemd-journal
08:07:05 PM     0       241      1.20      0.00      0.00      28  systemd-udevd
08:07:05 PM     0       246      0.09      0.00      0.00     162  loop0
08:07:05 PM     0       248      1.21      0.00      0.00     410  loop1
08:07:05 PM     0       249      0.04      0.00      0.00     141  loop2
08:07:05 PM     0       250      0.04      0.00      0.00     136  loop3
08:07:05 PM   104       446      0.05      0.00      0.00      28  systemd-timesyn
08:07:05 PM     0       449      0.01      0.00      0.00      18  cron
08:07:05 PM   103       450      0.06      0.00      0.00      92  dbus-daemon
08:07:05 PM     0       455      0.12      0.00      0.00     126  qemu-ga
08:07:05 PM     0       456      0.11      0.04      0.00      84  rsyslogd
08:07:05 PM     0       457      2.01      0.18      0.00     608  snapd
08:07:05 PM     0       459      0.02      0.00      0.00      21  systemd-logind
08:07:05 PM   108       462      1.74      0.01      0.00     401  zabbix_agent2
08:07:05 PM     0       463      0.02      0.00      0.00      33  agetty
08:07:05 PM   106       538      4.35      0.00      0.00     645  mariadbd
08:07:05 PM     0       565      0.66      0.00      0.00       0  sshd
08:07:05 PM     0       575      0.37      0.00      0.00     463  apache2
08:07:05 PM     0      1296      5.93      2.93      0.12       0  bash
08:07:05 PM     0      2255      0.00      0.00      0.00       1  kworker/u2:0-events_u

Leave a Comment