If you use Outlook 2007 check out Scott Hanselman's post about this new patch that improves the performance. All the initial feedback is extremely positive about it:
http://www.hanselman.com:80/blog/SpeedingUpASlowOutlook2007.aspx
~bk