I missed the point about you having this issue on a physical machine.
I also noticed on my systems that opening SysConfig, and especially opening Package Manager, is an operation that is much slower than most others.
The Support Package has a builtin benchmarking tool for the database. Did you run that benchmark? Are the results ok?
If everything checks out then the only possible cause that is left standing is the performance of mod_perl + Apache on your Linux distro. It would be worth to try a different, more common distro (such us CentOS or Ubuntu) and trying to run mod_perl and Apache in 32-bit vs 64-bit modes to see if there are any differences.
I'm also tackling some performance issue with my OTRS system as I'm trying to upgrade from 3.2 to 3.3 on Windows and during my investigations I've noticed mod_perl has somewhat fallen out of grace, not just on Windows where it became borderline unusable. You may want to try mod_fastcgi instead.