A Dolphin’s Journal of Events

iSCSI on our server really helps out quite a bit. For those who don’t know, iSCSI on Softlayer basically works like a networked drive, except it’s setup for high speed, meant for things like databases and such.

So far, it’s been great for the large 2 GB stats tracker table on the rscript project, but hasn’t sped much of anything else up. Most of the other tables are fairly small, and in fact, I think it’s slowed them down a bit more then anything. But if it has, it’s only by a little bit, and not exactly noticeable.

iSCSI sucks for large transfers though [a little less then our current 350 GB sata drives], but rules if you have apps that seek around a lot, with < .5 ms on Softlayer for seeking. [Most drives have 10-20 ms seek time, worse near the end of the drive] Definitely beats out getting a 15k rpm disk, cost wise.

---------

So anyway, we ran into another problem that's been haunting us for a long while now. Disk cache. On Windows 2003, the OS will swap out applications to disk, so it can fill the disk cache with large files. This can totally kill the server performance when doing even simple copy operations with large files. It doesn't make a whole lot of sense why the OS tries to swap out RUNNING applications to try and speed up file access, and then causing the whole machine to slow down like heck.

All I needed to do was limit the Disk Cache to a specific size. It'd keep the disk cache useful, without the negative effect of swapping applications out.

After a lot of searching, I finally found what I needed. There's many applications which periodically check the cache size then trims it, but I wanted one that would force it. Turns out there's not a whole lot out there, but Windows Server 2003 [sp1+] allows for "SetSystemFileCacheSize" system method, which I found a application for. You can find the periodically trimming application, as well as the permanent trim application for Windows Server and Vista, here.

4 GBs of ram on the server, so I set the cache to 1 GB. The results were fantastic.

Copying our entire web directory, none of the applications stalled or slowed down, and RDP remained speedy, as a bonus, the transfer only took 5 minutes instead of 30. Guess the swapping out really affected the disk operation more then the disk cache was suppose to fix, ironically. Finally I was able to run our backup script without things slowing down. As well, the Capped.TV encoder doesn’t kill the system when dumping raw video.

Linux systems don’t typically need to worry about this, since the disk cache is a bit smarter, according to some *nix admins I talked to.

Thanks to Cynthia for continuously telling me it was disk cache slowing things down which I didn’t listen to till after I eliminated every other variable. Sorry for not listening. <3

This entry was posted on Saturday, April 26th, 2008 at 12:27 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.