“sudo iosnoop” for slowdowns – the most useful command on OSX?

OSX has disgusting VM / management performance – once it’s started creating swapfiles, dynamically, it never lets them go and it manages them awfully badly.

My machine has been crawling for the past few days and nothing obvious is eating CPU. Remembering the existence of the Dtrace toolkit on OSX, I did:

sudo iosnoop

…and it instantly showed me that Firefox (which I was trying to shut down) and Skype (likewise) were hitting various swapfiles. I wasn’t expecting to get that information, so I was pleasantly surprised.

The overall diagnosis of the problem was pretty easy too:

du -kh /var/vm

…said I was using 4Gb of (probably rather fragmented) swapfile space.

A swift reboot and everything is OK again… for the moment… but I wonder whether I can create a OSX swap partition and use it preferentially to anything else, and I further wonder what else I can do to alleviate this.

Alas, OSX seems to be heading down towards where Windows used to be.

Comments

13 responses to ““sudo iosnoop” for slowdowns – the most useful command on OSX?”

  1. Good post, I have seen the same behavior but didn’t know about iosnoop. One more tip for the bag-o-tricks!

  2. Craig

    Along similar lines, when you see the vm utilisation climbing, its very often in cross-platform apps, typically they tend to leak memory where they seem to bridge to OSX libraries/APIs.

    Try `leaks ` out of interest on a few apps of choice and you’ll see where much of the memory disappears to. Significant offenders recently seem to include Adium, Tweetie and a few others.

  3. Craig

    Oops, looks like your comments filtered my syntax, I should have escaped my terms. That should read:

    # leaks PID

    😉

    1. Craig: very cool. Any idea what this might be?

      22:32:41 luther:~ # leaks 6305
      leaks cannot examine process 6305 for unknown reasons, even though it appears to exist. Try running with sudo.
      22:32:43 luther:~ # ps auxww | grep 6305
      alecm 6305 0.0 0.0 0 0 ?? Z 2:31pm 0:00.00 (Chromium Helper)

      …Chromium switches off DTrace, perhaps?

  4. Craig

    Oh, thats interesting … hadn’t tried Chromium for leaks … maybe its process model?

    BTW, think leaks predates DTrace in OSX by a few releases, so assume its instrumenting malloc routines by traditional methods.

    Odd, latest Chromium nightly seems OK on 10.6.2 with leaks, don’t observe that behaviour, reacts like other apps.

  5. Craig

    BTW, whilst iosnoop is great for visualising raw IO, OSX has lifted fsevents framework from BSD et al and there is a sweet graphical (freeware) utility called fseventer (http://fernlightning.com/doku.php?id=software:fseventer:start) which tracks traditional FS events.

    Very nice, clean way of tracking IO at a high-level and then revert to DTrace for lower level, ie. block size, operation, etc

    There are some great utilities available or hidden in OSX, its a pity more aren’t better documented … Instruments for instance is an inspired DTrace front-end, which Sun would never have come up with, whilst in standalone tools, Shark is particularly good.

    Have to blog some of these over Xmas …

  6. You may find this useful:

    http://www.macosxhints.com/article.php?story=20001215021440138

    It’s not a swap partition as such, but what you could do is create an HFS+ partition with a swap file upon it and tell the system swap on it at boot time.

    Of course, by making the file the same size as the disk space available using mkfile(8) it shouldn’t get fragmented.

  7. I’ve done some more investigation.

    That’s you’re looking for to control swap is:

    dynamic_pager(8)

    Basically, that organises the the kernel’s swap system. By default it creates 68MB files in /private/var/vm/ named swapfileN (where N is an integer) creating a new one when the previous one runs out of space.

    The 4GB you are seeing when you do a du(1) is actually the sleepimage file, e.g. on my MBP:

    bash-3.2# ls -lh
    total 8650752
    -rw——T 1 root wheel 4.0G Dec 23 16:37 sleepimage
    -rw——- 1 root wheel 64M Dec 23 08:39 swapfile0
    -rw——- 1 root wheel 64M Dec 23 17:10 swapfile1

    I imagine that if you know that you will be using a lot of swap you could tell dynamic_pager to use rather larger swapfiles, such as 4GB, which would then mean that it wouldn’t keep having to do filesystem look-ups when applications are fragmented between swapfiles.

    1. >The 4GB you are seeing when you do a du(1) is actually the sleepimage file, e.g. on my MBP:

      Actually, Steve, no; my MacMini never sleeps. What I am seeing when I ‘du’ is perhaps 8 or 9 swapfiles numbered 1 thru N, and which start multiplying in size after the first two. Eventually they get up to 1G per file.

      The files are deleted on reboot, they are fragmented, and they are journalled, which makes little sense.

      1. ps: keep yr mac awake for maybe a month and run virtualbox, firefox and a bunch of other apps. That’s when you see it.

  8. Hmm.. I may be useful to create a separate partition for the swapfiles then. Format it as a non-journalled HFS+ volume my help as well. There may be a way to persuade dynamic_pager(8) to be started with a large swapfile size to begin with, rather than starting small.

    I’ve not seen any hacks to get Snow Leopard to use swapfiles on a separate volume yet. It seems that there’s no /private/etc/rc script any more which could make things more difficult.

    1. i found one. it also suggests using UFS as fast-but-correct, and hacking the invocation of dynamic_pager

Leave a Reply

Your email address will not be published. Required fields are marked *