Recently I have notices that log files on my server grow faster than I was expecting. After a quick look I have realized that it is wtmp what aggressively is taking my disk space. Using utmpdump command (see below) I found out that every 5 seconds new 3 or 4 logs are recorded.
# utmpdump /var/log/wtmp | tail -n 25 Utmp dump of /var/log/wtmp [6] [00886] [2 ] [LOGIN ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:08 2018 MSK] [8] [00885] [1 ] [ ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:13 2018 MSK] [6] [00889] [1 ] [LOGIN ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:13 2018 MSK] [8] [00886] [2 ] [ ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:13 2018 MSK] [6] [00890] [2 ] [LOGIN ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:13 2018 MSK] [8] [00889] [1 ] [ ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:18 2018 MSK] [6] [00897] [1 ] [LOGIN ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:18 2018 MSK] [8] [00890] [2 ] [ ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:18 2018 MSK] [6] [00898] [2 ] [LOGIN ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:18 2018 MSK] [8] [00897] [1 ] [ ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:23 2018 MSK] [6] [00899] [1 ] [LOGIN ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:23 2018 MSK] [8] [00898] [2 ] [ ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:23 2018 MSK] [6] [00900] [2 ] [LOGIN ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:23 2018 MSK] [8] [00899] [1 ] [ ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:28 2018 MSK] [6] [00901] [1 ] [LOGIN ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:28 2018 MSK] [8] [00900] [2 ] [ ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:28 2018 MSK] [6] [00902] [2 ] [LOGIN ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:28 2018 MSK] [8] [00901] [1 ] [ ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:33 2018 MSK] [6] [00906] [1 ] [LOGIN ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:33 2018 MSK] [8] [00902] [2 ] [ ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:33 2018 MSK] [6] [00907] [2 ] [LOGIN ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:33 2018 MSK] [8] [00906] [1 ] [ ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:38 2018 MSK] [6] [00910] [1 ] [LOGIN ] [tty1 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:38 2018 MSK] [8] [00907] [2 ] [ ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:38 2018 MSK] [6] [00911] [2 ] [LOGIN ] [tty2 ] [ ] [0.0.0.0 ] [Wed Feb 07 17:26:38 2018 MSK]
There is no load on the server:
# w 17:34:03 up 17 min, 1 user, load average: 0.00, 0.00, 0.00 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT root pts/2 cpe-75-177-130-5 17:24 0.00s 0.02s 0.00s w
And no strange processes ruining:
# top
top - 17:35:08 up 18 min, 1 user, load average: 0.00, 0.00, 0.00
Tasks: 28 total, 1 running, 27 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.3%us, 0.0%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2097152k total, 47060k used, 2050092k free, 0k buffers
Swap: 0k total, 0k used, 0k free, 28024k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1141 root 20 0 11452 3536 2724 S 1.3 0.2 0:00.11 sshd
1 root 20 0 2844 1440 1228 S 0.0 0.1 0:00.27 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd/9506
3 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khelper/9506
72 root 16 -4 2560 600 364 S 0.0 0.0 0:00.00 udevd
98 root 18 -2 2556 604 364 S 0.0 0.0 0:00.00 udevd
99 root 18 -2 2556 604 364 S 0.0 0.0 0:00.00 udevd
458 root 20 0 9400 1008 520 S 0.0 0.0 0:00.02 sshd
469 root 20 0 3144 940 760 S 0.0 0.0 0:00.00 xinetd
483 root 20 0 6224 576 264 S 0.0 0.0 0:00.00 vsftpd
494 root 20 0 8704 864 468 S 0.0 0.0 0:00.00 saslauthd
496 root 20 0 8704 552 156 S 0.0 0.0 0:00.00 saslauthd
514 root 20 0 12352 1820 708 S 0.0 0.1 0:00.01 sendmail
521 smmsp 20 0 12152 1624 644 S 0.0 0.1 0:00.00 sendmail
533 root 20 0 25096 6956 3932 S 0.0 0.3 0:00.03 httpd
543 root 20 0 1964 496 436 S 0.0 0.0 0:00.00 mingetty
544 root 20 0 1964 488 436 S 0.0 0.0 0:00.00 mingetty
552 root 20 0 1964 492 436 S 0.0 0.0 0:00.00 mingetty
554 root 20 0 1964 488 436 S 0.0 0.0 0:00.00 mingetty
556 root 20 0 1964 492 436 S 0.0 0.0 0:00.00 mingetty
558 root 20 0 1964 492 436 S 0.0 0.0 0:00.00 mingetty
559 apache 20 0 25096 3676 628 S 0.0 0.2 0:00.00 httpd
831 root 20 0 12572 3652 2908 S 0.0 0.2 0:00.06 sshd
833 root 20 0 6372 1712 1472 S 0.0 0.1 0:00.02 bash
1136 root 20 0 2548 1076 892 R 0.0 0.1 0:00.00 top
1142 sshd 20 0 10744 1452 876 S 0.0 0.1 0:00.01 sshd
1145 root 20 0 1960 592 532 S 0.0 0.0 0:00.00 mingetty
1146 root 20 0 1960 596 532 S 0.0 0.0 0:00.00 mingetty
What is behind these log records and why such tasks are recorded every 5 seconds? Is there a way to stop record those “dummy” logs and have only real login logs recorded?
Advertisement
Answer
Record all processes running during 50 seconds
for i in {1..10} ; do ps -efH | tee -a ~/tmp/pids-5.txt; sleep 5; done
Then dump wtmp contents and check second column values against pids-5.txt. It should tell you which user and command the PID belongs to. You could then do something to avoid those process running.