Strange slowdowns (apache configuration?)
On 9/7/04 7:44 AM, "Duarte Cordeiro" wrote:
Latelly we¹ve been experiencing huge slowdowns on the server running OTRS. It only runs OTRS and nothing more, so there is no other process that may be using mem/disc/cpu.
-- Wes Plate Automatic Duck, Inc. http://www.wesplate.com http://www.automaticduck.com
On 9/7/04 7:44 AM, "Duarte Cordeiro" wrote:
Latelly we¹ve been experiencing huge slowdowns on the server running OTRS. It only runs OTRS and nothing more, so there is no other process that may be using mem/disc/cpu.
Sounds kind of like my problem in the thread... [otrs] Trying to send a reply, otrs doesn't respond -- Wes Plate Automatic Duck, Inc. http://www.wesplate.com http://www.automaticduck.com
Which database backend are you using?
Latelly we've been experiencing huge slowdowns on the server running OTRS. It only runs OTRS and nothing more, so there is no other process that may be using mem/disc/cpu.
15:20:35 up 20 days, 3:19, 1 user, load average: 78.75, 101.80, 87.94 189 processes: 186 sleeping, 3 running, 0 zombie, 0 stopped CPU states: cpu user nice system irq softirq iowait idle total 1.7% 0.0% 13.5% 4.9% 0.0% 79.7% 0.0% Mem: 384200k av, 379252k used, 4948k free, 0k shrd, 1180k buff 280884k actv, 67428k in_d, 1292k in_c Swap: 915696k av, 722552k used, 193144k free 11108k cached
The problem is, as you can see, with the memory usage. Sorting the top output by mem usage, I can see that I have lots of httpd processes each with 20M of mem usage.
Doing a ps auxww | grep httpd | wc I get 133 processes running.
The only way to solve this problem is to restart the apache server. Does it has anything to do with session timeout? Anyone experimented similar problems ? Solutions ?
Thanks in advance,
Duarte Cordeiro
_______________________________________________ OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/cgi-bin/listinfo/otrs Support oder Consulting für Ihr OTRS System?
-------------------------------------------------------- This message was sent using MetroWEB's AirMail service. http://www.metroweb.co.za/ - full access for only R73. Free Web Accelerator, WebMail, Calendar, Anti-Virus, Anti-Spam, 10 emails, 100MB personal webspace, and more! Phone Now! 086 11 11 440
Forgot about the stuff that matters(TM): Server running in a vmware ESX (hardware is a dual xenon 2.8Ghz, 6GB Ram, 200GB HD SCSI RAID5 on a DL580) VM has 400M memory allocated, 1 cpu 2.8Ghz, 10GB HD. VM running RedHat Enterprise Edition 3.0U1 OTRS 1.2.2 Mysql as backend DB on the same vm du -s /var/lib/mysql/otrs: 135M As I stated in the original post: - the number of httpd processes tend to rise over the time (starts with 10... in a day reachs easily 120) When this happens, the hardware CPU is at 5% , the emulated cpu at 70%, and the memory consumed is about 1.5GB inside the virtual machine. If the problem has anything to do with sessions being around too much time, the question is: where do I define the timeout ? in apache, in otrs ? Thanks, Duarte -----Original Message----- From: Henry Combrinck [mailto:henry@metroweb.co.za] Sent: terça-feira, 7 de Setembro de 2004 16:06 To: User questions and discussions about OTRS. Cc: duarte.cordeiro@cm-lisboa.pt Subject: Re: [otrs] Strange slowdowns (apache configuration?) (sorry for Which database backend are you using?
Latelly we've been experiencing huge slowdowns on the server running OTRS. It only runs OTRS and nothing more, so there is no other process that may be using mem/disc/cpu.
15:20:35 up 20 days, 3:19, 1 user, load average: 78.75, 101.80, 87.94 189 processes: 186 sleeping, 3 running, 0 zombie, 0 stopped CPU states: cpu user nice system irq softirq iowait idle total 1.7% 0.0% 13.5% 4.9% 0.0% 79.7% 0.0% Mem: 384200k av, 379252k used, 4948k free, 0k shrd, 1180k buff 280884k actv, 67428k in_d, 1292k in_c Swap: 915696k av, 722552k used, 193144k free 11108k cached
The problem is, as you can see, with the memory usage. Sorting the top output by mem usage, I can see that I have lots of httpd processes each with 20M of mem usage.
Doing a ps auxww | grep httpd | wc I get 133 processes running.
The only way to solve this problem is to restart the apache server. Does it has anything to do with session timeout? Anyone experimented similar problems ? Solutions ?
Thanks in advance,
Duarte Cordeiro
_______________________________________________ OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/cgi-bin/listinfo/otrs Support oder Consulting für Ihr OTRS System?
-------------------------------------------------------- This message was sent using MetroWEB's AirMail service. http://www.metroweb.co.za/ - full access for only R73. Free Web Accelerator, WebMail, Calendar, Anti-Virus, Anti-Spam, 10 emails, 100MB personal webspace, and more! Phone Now! 086 11 11 440
On Tuesday, September 07, 2004 4:49 PM
Duarte Cordeiro
The problem is, as you can see, with the memory usage. Sorting the top output by mem usage, I can see that I have lots of httpd processes each with 20M of mem usage. Doing a ps auxww | grep httpd | wc I get 133 processes running.
Everything quite normal. The number of processes has to somewhat correlate with the number of agents. What you can do: a) Buy more RAM. b) Restrict the maximum amount of prefork childs: <IfModule prefork.c> StartServers 10 MinSpareServers 20 MaxSpareServers 40 MaxClients 120 MaxRequestsPerChild 400 </IfModule> c) Switch to another MPM, say 'worker': <IfModule worker.c> StartServers 2 MaxClients 150 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0 </IfModule> d) Disable any module not needed, esp. mod_php4 e) Disable Apache::Reload in apache2-perl-startup.pl f) Buy more RAM. ;) g) Switch off mod_perl. Um, Bad idea - better buy more RAM. Es grüßt fröhlich Robert Kehl -- Ich habe zwei verschiedene Halbzeiten gesehen. Volker Finke
a) Buy more RAM.
The server has 6GB Ram, and the VM 400MB. I can easily ask for a increment on the VM, but how am I going to explain to the admins, that this webserver, serving 100 users (not concurrent users) needs more than 400MB of ram (only for running apache)?
b) Restrict the maximum amount of prefork childs: <IfModule prefork.c> StartServers 10 MinSpareServers 20 MaxSpareServers 40 MaxClients 120 MaxRequestsPerChild 400 </IfModule>
Done
c) Switch to another MPM, say 'worker': <IfModule worker.c> StartServers 2 MaxClients 150 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0 </IfModule>
I have both sections uncommented. How do I know which MPM is being used?
d) Disable any module not needed, esp. mod_php4 don't have any line with LoadModule mod_php4 or similar.
e) Disable Apache::Reload in apache2-perl-startup.pl that line doesn't exists in my file:
#use Apache (); #use Apache::DBI (); #Apache::DBI->connect_on_init('DBI:mysql:otrs', 'otrs', 'some-pass'); use DBI (); use DBD::mysql (); Thanks for your help. I've changed the number of max requests per child, and I'm waiting to see if it lags again :)
On Tuesday, September 07, 2004 6:38 PM
Duarte Cordeiro
a) Buy more RAM.
The server has 6GB Ram, and the VM 400MB. I can easily ask for a increment on the VM, but how am I going to explain to the admins, that this webserver, serving 100 users (not concurrent users) needs more than 400MB of ram (only for running apache)?
Let's calc: You've got 400 Megs. Say, 40 Megs are in use by the system. The remaining 360 Megs allow 360/20 = 18 requests coming in at the same time, and -boom- off goes the RAM. This is because Apache when being armed with mod_perl pre-loads every perl script used. Ie., every instances carries a complete and whole OTRS installation and code with itself.
c) Switch to another MPM, say 'worker': <IfModule worker.c> StartServers 2 MaxClients 150 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0 </IfModule>
I have both sections uncommented. How do I know which MPM is being used?
I guess Apache was installed as an RPM, so I guess the standard MPM Prefork was used. This would allow later loading of PHP, which isn't possibly with the Worker MPM. So, use your software package management software to switch the Apache version used from mpm_prefork to mpm_worker. The software will tell you which one is installed. Something like this should do so, too, iirc: # rpm -qa | grep apache
d) Disable any module not needed, esp. mod_php4 don't have any line with LoadModule mod_php4 or similar.
modules/enabled and stuff?
e) Disable Apache::Reload in apache2-perl-startup.pl that line doesn't exists in my file:
Sorry, it's scripts/apache2-httpd.include.conf With kind regards, Robert Kehl -- Erste Pass, gleich Scheiße! Dragoslav Stepanovic (beim ersten Training nach seinem Comeback in Frankfurt)
-----Original Message----- From: otrs-bounces@otrs.org [mailto:otrs-bounces@otrs.org] On Behalf Of Robert Kehl Sent: terça-feira, 7 de Setembro de 2004 17:55 To: User questions and discussions about OTRS. Subject: Re: [otrs] Strange slowdowns (apache configuration?) (sorryfortheHTMLpost)
Let's calc: You've got 400 Megs. Say, 40 Megs are in use by the system. The remaining 360 Megs allow 360/20 = 18 requests coming in at the same time, and -boom- off goes the RAM.
This is because Apache when being armed with mod_perl pre-loads every perl script used. Ie., every instances carries a complete and whole OTRS installation and code with itself.
I agree with this, but... today each process started using 18M and are now up to 28M... number of processes is stable right now (42), but memory increases over time (+200M per hour). This only happens for the last 4 or 5 days, so I suspect a memory leak caused by some bad article/ticket/or something else. But there is no malloc/free on perl (correct me if I'm wrong), so no point of blaming otrs... or apache... or mysql ... or even me :P It's getting to a point where I need to restart apache every couple of hours :( Any hints on what to do next?
On Wednesday, September 08, 2004 4:23 PM
Duarte Cordeiro
It's getting to a point where I need to restart apache every couple of hours :( Any hints on what to do next?
On Tuesday, September 07, 2004 5:21 PM
Duarte Cordeiro
Server running in a vmware ESX (hardware is a dual xenon 2.8Ghz, 6GB Ram, 200GB HD SCSI RAID5 on a DL580)
Can it be you're running out of disk space? With kind regards, Robert Kehl
No, free disk space is around 4GB.
-----Original Message-----
From: otrs-bounces@otrs.org [mailto:otrs-bounces@otrs.org] On Behalf Of Robert Kehl
Sent: quinta-feira, 9 de Setembro de 2004 1:26
To: User questions and discussions about OTRS.
Subject: Re: [otrs] Strange slowdowns (apacheconfiguration?)(sorryfortheHTMLpost)
On Wednesday, September 08, 2004 4:23 PM
Duarte Cordeiro
It's getting to a point where I need to restart apache every couple of hours :( Any hints on what to do next?
On Tuesday, September 07, 2004 5:21 PM
Duarte Cordeiro
Server running in a vmware ESX (hardware is a dual xenon 2.8Ghz, 6GB Ram, 200GB HD SCSI RAID5 on a DL580)
Can it be you're running out of disk space? With kind regards, Robert Kehl _______________________________________________ OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/cgi-bin/listinfo/otrs Support oder Consulting für Ihr OTRS System? => http://www.otrs.de/
Try watching your apache with apachetop: http://clueful.shagged.org/apachetop/ You might get a clue what's going wrong from that. Search your logs then. Perhaps you want to create additional, more comprehensive logs. hth, Robert Kehl -- Damals hat die halbe Nation hinter dem Fernseher gestanden. Franz Beckenbauer (über das WM-Finale 1990)
participants (4)
-
Duarte Cordeiro -
Henry Combrinck -
Robert Kehl -
Wes Plate