Logs under /var/stm/logs/os in HPUX

Ever wondered why /var/stm/logs/os is taking up huge space in /var mount point of HPUX? In this post we will see details about logs under this directory and how to handle them.

Most of the times you get /var getting full alerts from your monitoring system or you observe /var is filling up. This is normal behavior since most of the logs resides in /var and if some logs are growing fast they cause /var filling up fast. As a first troubleshooting step, you need to check big size files in /var.

Many times you see one of the culprit is /var/stm/logs/os directory. If you see inside this directory you will see something like below :

Shell

1

2

3

4

5

6

7

8

9

10

11

12

# ls -lrt /var/stm/logs/os

total 6986016

-rw-r--r--   1 root       root        512656 Apr 10  2008 log1.raw

-rw-r--r--   1 root       root        512656 Apr 10  2008 log2.raw

-rw-r--r--   1 root       root        512656 Apr 10  2008 log3.raw

-rw-r--r--   1 root       root        512656 Apr 11  2008 log4.raw

-rw-r--r--   1 root       root        512656 Apr 11  2008 log5.raw

-rw-r--r--   1 root       root        512656 Apr 11  2008 log6.raw

-rw-r--r--   1 root       root        512656 Apr 11  2008 log7.raw

----- ouput clipped -----

There are lots of raw log files taking up huge space collectively.

What are these logs under /var/stm/logs/os :

Your next question will be what are these files? what is purpose of these files on server?

These are raw files which are being logged and used by STM i.e. support tool manager. Those are logs collected by STM which has information about your hardware issues. By the above output you can see those are rotated when one log file cross certain file size. While rotating they are sequentially numbered. This numbering makes it easy when it comes to manage those logs.

How to read these logs :

You can read these logs using log viewer by STM. Goto CSTM console using command cstm –

Shell

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

# /usr/sbin/cstm

Running Command File (/usr/sbin/stm/ui/config/.stmrc).

-- Information --

Support Tools Manager

Version A.59.05

Product Number B4708AA

(C) Copyright Hewlett Packard Co. 1995-2007

All Rights Reserved

Use of this program is subject to the licensing restrictions described

in "Help-->On Version".  HP shall not be liable for any damages resulting

from misuse or unauthorized use of this program.

cstm>

Then run ru and select logtool utility.

Shell

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

cstm>ru

-- Run Utility --

Select Utility

    1 MOutil

    2 logtool

Enter selection or cancel to quit : 2

-- Logtool Utility --

To View a Summary of Events in a Raw Log

  1. Select a raw (unformatted) log file.  (File Menu -> "Select Raw")

     The current log file ends in ".cur", e.g. "log1.raw.cur".

     You do not have to switch logs.

  2. View the summary of the selected log file. (View Menu -> "Raw Summary")

To Format a Raw Log

  1. Set the format filter for the types of entries you want to see.

     (Filters Menu -> "Format").  To see all entries, skip this step.

  2. Format the raw log file. (File Menu -> "Format Raw")

  3. Display the formatted file. (View Menu -> "Formatted Log")

  4. To further narrow the entries displayed, set a display filter.

     (Filters Menu -> "Display" -> "Formatted")

For more information, use the on-line help (Help Menu -> "General help").

Logtool Utility>

With give information on console you can view, format raw log files.

Should I purge or zip /var/stm/logs/os logs ?

Now you know what are these files and you observe there are too many of them which are too old to keep. In such scenario, you have got two options:

Zip them : For few months old files. May be 1-2 months old. How to Zip files.
Purge them : For very old logs like 6 or more months old.

Make a note that those logs are read by STM as well so if you purge or zip them, STM wont be able to use them.

So be sure you check logs using logtool utility explained above and decide to purge, zip or keep it. Normally, if you are not facing any hardware issues with server currently then you should zip/purge according to time frame I suggested above.

Shell

1

2

3

4

5

6

7

8

9

10

11

# ls -lrt /var/stm/logs/os

total 2463008

-rw-r--r--   1 root       root         65910 Apr 10  2008 log1.raw.gz

-rw-r--r--   1 root       root         57168 Apr 10  2008 log2.raw.gz

-rw-r--r--   1 root       root         53727 Apr 10  2008 log3.raw.gz

-rw-r--r--   1 root       root         40526 Apr 11  2008 log4.raw.gz

-rw-r--r--   1 root       root         39541 Apr 11  2008 log5.raw.gz

-rw-r--r--   1 root       root         37050 Apr 11  2008 log6.raw.gz

-rw-r--r--   1 root       root         37624 Apr 11  2008 log7.raw.gz

Match above output with previous and see how file size decreased after zipping which in turns saved my /var space.

Zipping of purging these logs will greatly free up space under /var mount point. This is one of the directory which we normally miss or ignore while cleaning up mount point.

Alternatively you can even Configure logrotate utility which will take care of this zipping and purging of files automatically without human intervention.