
Hello. I've installed otrs v.5 on a physical server with apache and mod_perl2. The backend DB is Mysql on different physical server in same LAN. The problem is with slow page opening. Every page takes 15-20 second to open: Tickets, Dashboard, Customers, Admin. I've monitored servers and nor Mysql server, nor apache server is fully loaded during pages navigation. How can I debug the problem and speed up the otrs? Thank you

Have you seen https://otrs.github.io/doc/manual/admin/5.0/en/html/performance-tuning.html already? Sander

Yes. I've read that article. There are merely 150 tickets. So this article not really address my problem. Cache has only 147 MB. There are only me for now using otrs to test. On 30.06.2016 18:01, Sander Goudswaard wrote:
Have you seen https://otrs.github.io/doc/manual/admin/5.0/en/html/performance-tuning.html already?
Sander
--------------------------------------------------------------------- OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/mailman/listinfo/otrs

Hi, same here. I'm currently investigating this issue as OTRS performance log looks fine. I'll share my results as soon I've a clue what's happening. Just to make sure: Do you make heavy use of ACLs and maybe disabled caching for debugging purposes? Regards, Yves Von meinem iPhone gesendet
Am 30.06.2016 um 17:09 schrieb Mimiko
: Yes. I've read that article.
There are merely 150 tickets. So this article not really address my problem.
Cache has only 147 MB.
There are only me for now using otrs to test.
On 30.06.2016 18:01, Sander Goudswaard wrote: Have you seen https://otrs.github.io/doc/manual/admin/5.0/en/html/performance-tuning.html already?
Sander
--------------------------------------------------------------------- OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/mailman/listinfo/otrs
--------------------------------------------------------------------- OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/mailman/listinfo/otrs
------------------------------------------------------- >>> business. people. technology. <<< ------------------------------------------------------- adesso AG mit Sitz in Dortmund Vorstand: Michael Kenfenheuer (Vors.), Christoph Junge, Andreas Prenneis Vorsitzender des Aufsichtsrates: Prof. Dr. Volker Gruhn Amtsgericht Dortmund HRB 20663

So the results:
ab -n 100 -c 2 http://example.com/otrs-web/skins/Agent/default/css/thirdparty/fontawesome/f... This is ApacheBench, Version 2.3 <$Revision$> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking example.com (be patient).....done Server Software: Apache/2.4.7 Server Hostname: example.com Server Port: 80 Document Path: /otrs-web/skins/Agent/default/css/thirdparty/fontawesome/fontawesome-webfont.svg Document Length: 313398 bytes Concurrency Level: 2 Time taken for tests: 0.317 seconds Complete requests: 100 Failed requests: 0 Total transferred: 31376400 bytes HTML transferred: 31339800 bytes Requests per second: 315.12 [#/sec] (mean) Time per request: 6.347 [ms] (mean) Time per request: 3.173 [ms] (mean, across all concurrent requests) Transfer rate: 96556.71 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 4 6 1.7 6 13 Waiting: 0 4 2.2 4 11 Total: 4 6 1.8 6 13 Percentage of the requests served within a certain time (ms) 50% 6 66% 6 75% 7 80% 7 90% 8 95% 11 98% 12 99% 13 100% 13 (longest request) ------------------- From another server:
ab -n 100 -c 2 http://example.com/otrs-web/skins/Agent/default/css/thirdparty/fontawesome/f... This is ApacheBench, Version 2.3 <$Revision: 1373084 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking example.com (be patient).....done Server Software: Apache/2.4.7 Server Hostname: example.com Server Port: 80 Document Path: /otrs-web/skins/Agent/default/css/thirdparty/fontawesome/fontawesome-webfont.svg Document Length: 313398 bytes Concurrency Level: 2 Time taken for tests: 0.689 seconds Complete requests: 100 Failed requests: 0 Write errors: 0 Total transferred: 31376400 bytes HTML transferred: 31339800 bytes Requests per second: 145.24 [#/sec] (mean) Time per request: 13.771 [ms] (mean) Time per request: 6.885 [ms] (mean, across all concurrent requests) Transfer rate: 44502.27 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 1 1.1 0 8 Processing: 8 13 4.4 12 38 Waiting: 3 6 3.4 4 26 Total: 9 14 4.6 13 38 Percentage of the requests served within a certain time (ms) 50% 13 66% 13 75% 15 80% 16 90% 20 95% 23 98% 29 99% 38 100% 38 (longest request) ------------------------------------
time mysql --host=ip --user=user --password=password -e 'select count(tn) from ticket;' otrs +-----------+ | count(tn) | +-----------+ | 139 | +-----------+
real 0m0.010s user 0m0.004s sys 0m0.000s ------------------------
time mysql --host=ip --user=user --password=password -e "SELECT BENCHMARK(1000,(SELECT t.tn FROM queue AS q, ticket AS t, article AS a, article_type AS a_t, ( SELECT _a.ticket_id, MAX(_a.create_time) AS create_time FROM ticket AS _t, ticket_state AS _t_s, article AS _a WHERE _t_s.name IN ('new', 'open') AND _t.ticket_state_id = _t_s.id AND _a.ticket_id = _t.id GROUP BY _a.ticket_id ) a_max WHERE q.id = t.queue_id AND t.id = a_max.ticket_id AND a.create_time = a_max.create_time AND a_t.id = a.article_type_id GROUP BY t.id ORDER BY a.create_time DESC LIMIT 0,1));" otrs +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | BENCHMARK(1000,(SELECT t.tn FROM queue AS q, ticket AS t, article AS a, article_type AS a_t, ( SELECT _a.ticket_id, MAX(_a.create_time) AS create_time FROM ticket AS _t, ticket_state AS _t_s, article AS _a WHERE _t_s.name IN ('new', 'open') AND _t.ticket_ | +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
0 | +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ real 0m0.013s user 0m0.004s sys 0m0.004s This seems ok. So what could it be?

Hi, please try to run the "ab" from your client which is expecting the poor performance. If you run it from another server you should also try to browse OTRS from this server (cURL, wget, lynx) to make sure that the performance issues are not L3 related. Otherwise the results aren't comparable. On the OTRS server please run: # time sudo -uotrs ./otrs.Console.pl Maint::Database::Check # sudo -uotrs ./otrs.Console.pl Dev::Tools::CacheBenchmark I'm not sure if unreachability could lead to bad user experience but you could also try to check if cloud services are reachable and not blocked by firewall. # sudo -uotrs ./otrs.Console.pl Maint::CloudServices::ConnectionCheck Regards, Yves
On 01 Jul 2016, at 09:55, Mimiko
wrote: So the results:
ab -n 100 -c 2 http://example.com/otrs-web/skins/Agent/default/css/thirdparty/fontawesome/f... This is ApacheBench, Version 2.3 <$Revision$> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking example.com (be patient).....done
Server Software: Apache/2.4.7 Server Hostname: example.com Server Port: 80
Document Path: /otrs-web/skins/Agent/default/css/thirdparty/fontawesome/fontawesome-webfont.svg Document Length: 313398 bytes
Concurrency Level: 2 Time taken for tests: 0.317 seconds Complete requests: 100 Failed requests: 0 Total transferred: 31376400 bytes HTML transferred: 31339800 bytes Requests per second: 315.12 [#/sec] (mean) Time per request: 6.347 [ms] (mean) Time per request: 3.173 [ms] (mean, across all concurrent requests) Transfer rate: 96556.71 [Kbytes/sec] received
Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 1 Processing: 4 6 1.7 6 13 Waiting: 0 4 2.2 4 11 Total: 4 6 1.8 6 13
Percentage of the requests served within a certain time (ms) 50% 6 66% 6 75% 7 80% 7 90% 8 95% 11 98% 12 99% 13 100% 13 (longest request)
-------------------
From another server:
ab -n 100 -c 2 http://example.com/otrs-web/skins/Agent/default/css/thirdparty/fontawesome/f... This is ApacheBench, Version 2.3 <$Revision: 1373084 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking example.com (be patient).....done
Server Software: Apache/2.4.7 Server Hostname: example.com Server Port: 80
Document Path: /otrs-web/skins/Agent/default/css/thirdparty/fontawesome/fontawesome-webfont.svg Document Length: 313398 bytes
Concurrency Level: 2 Time taken for tests: 0.689 seconds Complete requests: 100 Failed requests: 0 Write errors: 0 Total transferred: 31376400 bytes HTML transferred: 31339800 bytes Requests per second: 145.24 [#/sec] (mean) Time per request: 13.771 [ms] (mean) Time per request: 6.885 [ms] (mean, across all concurrent requests) Transfer rate: 44502.27 [Kbytes/sec] received
Connection Times (ms) min mean[+/-sd] median max Connect: 0 1 1.1 0 8 Processing: 8 13 4.4 12 38 Waiting: 3 6 3.4 4 26 Total: 9 14 4.6 13 38
Percentage of the requests served within a certain time (ms) 50% 13 66% 13 75% 15 80% 16 90% 20 95% 23 98% 29 99% 38 100% 38 (longest request)
------------------------------------
time mysql --host=ip --user=user --password=password -e 'select count(tn) from ticket;' otrs +-----------+ | count(tn) | +-----------+ | 139 | +-----------+
real 0m0.010s user 0m0.004s sys 0m0.000s
------------------------
time mysql --host=ip --user=user --password=password -e "SELECT BENCHMARK(1000,(SELECT t.tn FROM queue AS q, ticket AS t, article AS a, article_type AS a_t, ( SELECT _a.ticket_id, MAX(_a.create_time) AS create_time FROM ticket AS _t, ticket_state AS _t_s, article AS _a WHERE _t_s.name IN ('new', 'open') AND _t.ticket_state_id = _t_s.id AND _a.ticket_id = _t.id GROUP BY _a.ticket_id ) a_max WHERE q.id = t.queue_id AND t.id = a_max.ticket_id AND a.create_time = a_max.create_time AND a_t.id = a.article_type_id GROUP BY t.id ORDER BY a.create_time DESC LIMIT 0,1));" otrs +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | BENCHMARK(1000,(SELECT t.tn FROM queue AS q, ticket AS t, article AS a, article_type AS a_t, ( SELECT _a.ticket_id, MAX(_a.create_time) AS create_time FROM ticket AS _t, ticket_state AS _t_s, article AS _a WHERE _t_s.name IN ('new', 'open') AND _t.ticket_ | +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 0 | +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
real 0m0.013s user 0m0.004s sys 0m0.004s
This seems ok. So what could it be?
--------------------------------------------------------------------- OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/mailman/listinfo/otrs
------------------------------------------------------- >>> business. people. technology. <<< ------------------------------------------------------- adesso AG mit Sitz in Dortmund Vorstand: Michael Kenfenheuer (Vors.), Christoph Junge, Andreas Prenneis Vorsitzender des Aufsichtsrates: Prof. Dr. Volker Gruhn Amtsgericht Dortmund HRB 20663

* Vogl, Yves
Hi,
please try to run the "ab" from your client which is expecting the poor performance. If you run it from another server you should also try to browse OTRS from this server (cURL, wget, lynx) to make sure that the performance issues are not L3 related. Otherwise the results aren't comparable.
On the OTRS server please run:
# time sudo -uotrs ./otrs.Console.pl Maint::Database::Check # sudo -uotrs ./otrs.Console.pl Dev::Tools::CacheBenchmark
Which values are considered normal? # time sudo -uotrs /opt/otrs/bin/otrs.Console.pl Maint::Database::Check Trying to connect to database 'DBI:mysql:database=otrs;host=localhost;' with user 'otrs'... Connection successful. real 0m7.694s user 0m0.176s sys 0m0.012s # sudo -uotrs /opt/otrs/bin/otrs.Console.pl Dev::Tools::CacheBenchmark Testing cache module FileStorable Preloading cache with 100k x 1kB items... done. Cache module Item Size[b] Operations Time[s] Op/s Set OK Get OK Del OK --------------- ------------ ---------- ------- ------- ------- ------- ------- FileStorable 64 26000 1.51 17261 100.00% 100.00% 100.00% FileStorable 256 21000 1.22 17220 100.00% 100.00% 100.00% FileStorable 512 21000 1.22 17281 100.00% 100.00% 100.00% FileStorable 1024 16000 0.94 17100 100.00% 100.00% 100.00% FileStorable 4096 16000 0.97 16443 100.00% 100.00% 100.00% FileStorable 10240 11000 0.75 14663 100.00% 100.00% 100.00% FileStorable 102400 6000 0.97 6157 100.00% 100.00% 100.00% FileStorable 1048576 1000 2.35 425 100.00% 100.00% 100.00% FileStorable 4194304 1000 10.15 99 100.00% 100.00% 100.00% Removing preloaded 100k x 1kB items... done. -- Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebrandt@charite.de Campus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

Vogl, this are my results:
time sudo -uotrs ./otrs.Console.pl Maint::Database::Check Trying to connect to database 'DBI:mysql:database=otrs;host=10.10.x.x;' with user 'otrs'... Connection successful.
real 0m0.867s user 0m0.492s sys 0m0.068s -----------------------------------
sudo -uotrs ./otrs.Console.pl Dev::Tools::CacheBenchmark Testing cache module FileStorable Preloading cache with 100k x 1kB items... done. Cache module Item Size[b] Operations Time[s] Op/s Set OK Get OK Del OK
------- FileStorable 64 26000 3.99 6519 100.00% 100.00% 100.00% FileStorable 256 21000 3.96 5301 100.00% 100.00% 100.00% FileStorable 512 21000 3.06 6861 100.00% 100.00% 100.00% FileStorable 1024 16000 2.79 5739 100.00% 100.00% 100.00% FileStorable 4096 16000 2.57 6227 100.00% 100.00% 100.00% FileStorable 10240 11000 1.93 5704 100.00% 100.00% 100.00% FileStorable 102400 6000 2.88 2086 100.00% 100.00% 100.00% FileStorable 1048576 1000 7.22 139 100.00% 100.00% 100.00% FileStorable 4194304 1000 29.93 33 100.00% 100.00% 100.00% Removing preloaded 100k x 1kB items... done. ----------------------------
sudo -uotrs ./otrs.Console.pl Maint::CloudServices::ConnectionCheck Checking OTRS cloud service connectivity... Sending request with the following options: Timeout: 15 second(s)
Response time: 3 second(s) Response: 'Content' => \'{"ErrorMessage":"","Results":{"Test":[{"Data":{"Success":1},"Success":"1","Operation":"Test"}]},"Success":1}', 'Status' => '200 OK' -- Mimiko desu.

Did you try running "ab" from a client which is experiencing those "15 seconds" response times? Von meinem iPhone gesendet
Am 01.07.2016 um 15:27 schrieb Mimiko
: Vogl, this are my results:
time sudo -uotrs ./otrs.Console.pl Maint::Database::Check Trying to connect to database 'DBI:mysql:database=otrs;host=10.10.x.x;' with user 'otrs'... Connection successful.
real 0m0.867s user 0m0.492s sys 0m0.068s
-----------------------------------
sudo -uotrs ./otrs.Console.pl Dev::Tools::CacheBenchmark Testing cache module FileStorable Preloading cache with 100k x 1kB items... done. Cache module Item Size[b] Operations Time[s] Op/s Set OK Get OK Del OK
FileStorable 64 26000 3.99 6519 100.00% 100.00% 100.00% FileStorable 256 21000 3.96 5301 100.00% 100.00% 100.00% FileStorable 512 21000 3.06 6861 100.00% 100.00% 100.00% FileStorable 1024 16000 2.79 5739 100.00% 100.00% 100.00% FileStorable 4096 16000 2.57 6227 100.00% 100.00% 100.00% FileStorable 10240 11000 1.93 5704 100.00% 100.00% 100.00% FileStorable 102400 6000 2.88 2086 100.00% 100.00% 100.00% FileStorable 1048576 1000 7.22 139 100.00% 100.00% 100.00% FileStorable 4194304 1000 29.93 33 100.00% 100.00% 100.00% Removing preloaded 100k x 1kB items... done.
----------------------------
sudo -uotrs ./otrs.Console.pl Maint::CloudServices::ConnectionCheck Checking OTRS cloud service connectivity... Sending request with the following options: Timeout: 15 second(s)
Response time: 3 second(s)
Response: 'Content' => \'{"ErrorMessage":"","Results":{"Test":[{"Data":{"Success":1},"Success":"1","Operation":"Test"}]},"Success":1}', 'Status' => '200 OK'
-- Mimiko desu. --------------------------------------------------------------------- OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/mailman/listinfo/otrs
------------------------------------------------------- >>> business. people. technology. <<< ------------------------------------------------------- adesso AG mit Sitz in Dortmund Vorstand: Michael Kenfenheuer (Vors.), Christoph Junge, Andreas Prenneis Vorsitzender des Aufsichtsrates: Prof. Dr. Volker Gruhn Amtsgericht Dortmund HRB 20663

On 01.07.2016 16:35, Vogl, Yves wrote:
Did you try running "ab" from a client which is experiencing those "15 seconds" response times?
This are the results: ab -n 100 -c 2 http://example.com/otrs-web/skins/Agent/default/css/thirdparty/fontawesome/f... This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking ticket.acc.md (be patient).....done Server Software: Apache/2.4.7 Server Hostname: example.com Server Port: 80 Document Path: /otrs-web/skins/Agent/default/css/thirdparty/fontawesome/fontawesome-webfont.svg Document Length: 313398 bytes Concurrency Level: 2 Time taken for tests: 0.640 seconds Complete requests: 100 Failed requests: 0 Write errors: 0 Total transferred: 31376400 bytes HTML transferred: 31339800 bytes Requests per second: 156.13 [#/sec] (mean) Time per request: 12.810 [ms] (mean) Time per request: 6.405 [ms] (mean, across all concurrent requests) Transfer rate: 47839.36 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 1.6 0 16 Processing: 0 13 6.5 16 31 Waiting: 0 5 7.3 0 16 Total: 0 13 6.4 16 31 Percentage of the requests served within a certain time (ms) 50% 16 66% 16 75% 16 80% 16 90% 16 95% 16 98% 16 99% 31 100% 31 (longest request)

And you've constantly high response times (over 10s) when browsing OTRS from this client? Von meinem iPhone gesendet
Am 01.07.2016 um 17:00 schrieb Mimiko
: On 01.07.2016 16:35, Vogl, Yves wrote: Did you try running "ab" from a client which is experiencing those "15 seconds" response times?
This are the results:
ab -n 100 -c 2 http://example.com/otrs-web/skins/Agent/default/css/thirdparty/fontawesome/f... This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking ticket.acc.md (be patient).....done
Server Software: Apache/2.4.7 Server Hostname: example.com Server Port: 80
Document Path: /otrs-web/skins/Agent/default/css/thirdparty/fontawesome/fontawesome-webfont.svg Document Length: 313398 bytes
Concurrency Level: 2 Time taken for tests: 0.640 seconds Complete requests: 100 Failed requests: 0 Write errors: 0 Total transferred: 31376400 bytes HTML transferred: 31339800 bytes Requests per second: 156.13 [#/sec] (mean) Time per request: 12.810 [ms] (mean) Time per request: 6.405 [ms] (mean, across all concurrent requests) Transfer rate: 47839.36 [Kbytes/sec] received
Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 1.6 0 16 Processing: 0 13 6.5 16 31 Waiting: 0 5 7.3 0 16 Total: 0 13 6.4 16 31
Percentage of the requests served within a certain time (ms) 50% 16 66% 16 75% 16 80% 16 90% 16 95% 16 98% 16 99% 31 100% 31 (longest request) --------------------------------------------------------------------- OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/mailman/listinfo/otrs
------------------------------------------------------- >>> business. people. technology. <<< ------------------------------------------------------- adesso AG mit Sitz in Dortmund Vorstand: Michael Kenfenheuer (Vors.), Christoph Junge, Andreas Prenneis Vorsitzender des Aufsichtsrates: Prof. Dr. Volker Gruhn Amtsgericht Dortmund HRB 20663

On 01.07.2016 18:30, Vogl, Yves wrote:
And you've constantly high response times (over 10s) when browsing OTRS from this client?
Yes, always. Some pages have 3-5 sec. Especially if repetitive refresh page. And it not only one client problem. I am talking about agent's module, not client's interface. Although, that interface is simple, browsing takes 1-3 sec. Maybe perl is so slow?

Please enable OTRS performance logging and post some values. To go further I'd take a look at Apache VMonitor and correlate the results with some automated replays of requests from your client by using Charles Proxy. But it's pretty strange though. I'm not a big fan of perl for webapps anyway but in most cases it's good enough. I've setup OTRS 4 with 15 concurrent agents and up to 10 customers, over 160.000 tickets and with a lot of bells & whistles. The performance was quite ok (ca. 5req/s with 95% < 0.8s). Best performance results are achieved with mod_perl2 on a dedicated server and mod_fcgid in a shared environment. Apache MPM worker had a slightly smaller footprint than the prefork MPM. Von meinem iPhone gesendet
Am 01.07.2016 um 17:34 schrieb Mimiko
: On 01.07.2016 18:30, Vogl, Yves wrote: And you've constantly high response times (over 10s) when browsing OTRS from this client?
Yes, always. Some pages have 3-5 sec. Especially if repetitive refresh page. And it not only one client problem. I am talking about agent's module, not client's interface. Although, that interface is simple, browsing takes 1-3 sec.
Maybe perl is so slow? --------------------------------------------------------------------- OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/mailman/listinfo/otrs
------------------------------------------------------- >>> business. people. technology. <<< ------------------------------------------------------- adesso AG mit Sitz in Dortmund Vorstand: Michael Kenfenheuer (Vors.), Christoph Junge, Andreas Prenneis Vorsitzender des Aufsichtsrates: Prof. Dr. Volker Gruhn Amtsgericht Dortmund HRB 20663

On 01.07.2016 18:46, Vogl, Yves wrote:
Please enable OTRS performance logging and post some values.
For example: Range (last 5 m) Interface Requests Min Response Max Response Average Response Agent 41 0s 14s 2.34s Admin 3 1s 2s 1.33s AdminPerformanceLog 1 1s 1s 1s AdminSysConfig 1 5s 5s 5s AdminSysConfig&Search 1 12s 12s 12s AdminSysConfig&Update 1 14s 14s 14s AdminSysConfig;Subaction=Edit;SysConf... 1 12s 12s 12s AdminSysConfig;Subaction=Edit;SysConf... 1 6s 6s 6s AdminSysConfig;Subaction=Edit;SysConf... 1 10s 10s 10s AgentBook 1 1s 1s 1s AgentBook;ToCustomer=;CcCustomer=;Bcc... 2 1s 1s 1s AgentCustomerSearch 3 0s 1s 0.66s AgentDashboard 1 4s 4s 4s AgentFAQExplorer;CategoryID=1;Nav=None 1 1s 1s 1s AgentFAQExplorer;Nav=None;Subject=;What= 1 2s 2s 2s AgentFAQZoom;ItemID=2;Nav=None 1 1s 1s 1s AgentFAQZoom;Subaction=HTMLView;ItemI... 1 0s 0s 0s AgentFAQZoom;Subaction=HTMLView;ItemI... 1 0s 0s 0s AgentFAQZoom;Subaction=HTMLView;ItemI... 1 0s 0s 0s AgentFAQZoom;Subaction=HTMLView;ItemI... 1 1s 1s 1s AgentTicketAttachment;Subaction=HTMLV... 1 1s 1s 1s AgentTicketCompose 4 0s 1s 0.75s AgentTicketCompose&AJAXUpdate 1 1s 1s 1s AgentTicketLockedView 1 1s 1s 1s AgentTicketQueue 1 2s 2s 2s AgentTicketQueue;QueueID=0;View=Small... 1 1s 1s 1s AgentTicketQueue;QueueID=5;SortBy=Age... 1 2s 2s 2s AgentTicketQueue;QueueID=5;View=;Filt... 1 1s 1s 1s AgentTicketQueue;QueueID=7;View=;Filt... 1 1s 1s 1s AgentTicketStatusView 1 2s 2s 2s AgentTicketZoom&MarkAsSeen 2 0s 0s 0s AgentTicketZoom;TicketID=118 1 1s 1s 1s AgentTicketZoom;TicketID=177 1 2s 2s 2s -- Mimiko desu.

As far as I understand those values they target the internal processing time. So next step would be profiling the internal processing of request to get a trace where the time is spent. I'd fire up NYTProf for this. http://search.cpan.org/dist/Devel-NYTProf/lib/Devel/NYTProf/Apache.pm Von meinem iPhone gesendet
Am 01.07.2016 um 18:06 schrieb Mimiko
: On 01.07.2016 18:46, Vogl, Yves wrote: Please enable OTRS performance logging and post some values.
For example:
Range (last 5 m) Interface Requests Min Response Max Response Average Response Agent 41 0s 14s 2.34s Admin 3 1s 2s 1.33s AdminPerformanceLog 1 1s 1s 1s AdminSysConfig 1 5s 5s 5s AdminSysConfig&Search 1 12s 12s 12s AdminSysConfig&Update 1 14s 14s 14s AdminSysConfig;Subaction=Edit;SysConf... 1 12s 12s 12s AdminSysConfig;Subaction=Edit;SysConf... 1 6s 6s 6s AdminSysConfig;Subaction=Edit;SysConf... 1 10s 10s 10s AgentBook 1 1s 1s 1s AgentBook;ToCustomer=;CcCustomer=;Bcc... 2 1s 1s 1s AgentCustomerSearch 3 0s 1s 0.66s AgentDashboard 1 4s 4s 4s AgentFAQExplorer;CategoryID=1;Nav=None 1 1s 1s 1s AgentFAQExplorer;Nav=None;Subject=;What= 1 2s 2s 2s AgentFAQZoom;ItemID=2;Nav=None 1 1s 1s 1s AgentFAQZoom;Subaction=HTMLView;ItemI... 1 0s 0s 0s AgentFAQZoom;Subaction=HTMLView;ItemI... 1 0s 0s 0s AgentFAQZoom;Subaction=HTMLView;ItemI... 1 0s 0s 0s AgentFAQZoom;Subaction=HTMLView;ItemI... 1 1s 1s 1s AgentTicketAttachment;Subaction=HTMLV... 1 1s 1s 1s AgentTicketCompose 4 0s 1s 0.75s AgentTicketCompose&AJAXUpdate 1 1s 1s 1s AgentTicketLockedView 1 1s 1s 1s AgentTicketQueue 1 2s 2s 2s AgentTicketQueue;QueueID=0;View=Small... 1 1s 1s 1s AgentTicketQueue;QueueID=5;SortBy=Age... 1 2s 2s 2s AgentTicketQueue;QueueID=5;View=;Filt... 1 1s 1s 1s AgentTicketQueue;QueueID=7;View=;Filt... 1 1s 1s 1s AgentTicketStatusView 1 2s 2s 2s AgentTicketZoom&MarkAsSeen 2 0s 0s 0s AgentTicketZoom;TicketID=118 1 1s 1s 1s AgentTicketZoom;TicketID=177 1 2s 2s 2s
-- Mimiko desu. --------------------------------------------------------------------- OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/mailman/listinfo/otrs
------------------------------------------------------- >>> business. people. technology. <<< ------------------------------------------------------- adesso AG mit Sitz in Dortmund Vorstand: Michael Kenfenheuer (Vors.), Christoph Junge, Andreas Prenneis Vorsitzender des Aufsichtsrates: Prof. Dr. Volker Gruhn Amtsgericht Dortmund HRB 20663

Hello. I've setup NYTProf to start with apache. Clicked some pages. There where several nytprof.$$.out.pid files. Not all files were decrypted with nytprofhtml. I've found big seconds on two of files. I don't know what to search for very well, so this is some text:
Profile of /srv/httpd/bin/httpd for 40.4s (of 40.6s), executing 51799 statements and 39622 subroutine calls in 174 source files and 53 string evals. Top 15 Subroutines Calls P F Exclusive Time Inclusive Time Subroutine 1 1 1 3.00s 3.00s DBD::mysql::db::_login (xsub) 13711 1 1 571ms 619ms Encode::utf8::decode_xs (xsub) 1 1 1 383ms 1.00s Kernel::System::Fred::STDERRLog::CORE:readline (opcode) 29 8 7 222ms 498ms Kernel::System::Main::Require 1 1 1 114ms 142ms Kernel::Config::Defaults::new 25 1 1 73.3ms 782ms Kernel::System::ObjectManager::_ObjectBuild (recurses: max depth 1, inclusive time 157ms) 1 1 1 66.5ms 98.3ms Kernel::System::ObjectManager::BEGIN@24 4 1 1 49.0ms 49.9ms Template::Document::new 13711 1 1 47.8ms 47.8ms Encode::Encoding::renewed 1 1 1 46.3ms 46.4ms Template::Parser::BEGIN@41 1 1 1 41.7ms 1.05s Kernel::System::Fred::STDERRLog::DataGet 10 3 1 40.3ms 120ms Template::Config::load 4 4 4 36.2ms 39.8ms utf8::SWASHNEW 20 20 20 33.9ms 96.8ms base::import (recurses: max depth 1, inclusive time 135µs) 1 1 1 28.8ms 29.0ms Kernel::System::ObjectManager::BEGIN@30 See all 2571 subroutines
Source Code Files — ordered by exclusive time then name Stmts Exclusive Time Reports Source File 124 3.02s line • block • sub DBD/mysql.pm 114 1.05s line • block • sub /srv/otrs//Kernel/System/Fred/STDERRLog.pm 1 123ms line • block • sub -e 2338 112ms line • block • sub /srv/otrs/Kernel/Config/Files/ZZZAAuto.pm 716 76.2ms line • block • sub /srv/otrs//Kernel/Output/HTML/Layout.pm 2230 68.8ms line • block • sub /srv/otrs//Kernel/Language/en_ITSMChangeManagement.pm 1580 66.3ms line • block • sub /srv/otrs//Kernel/System/ObjectManager.pm 210 53.0ms line • block • sub Template/Document.pm (including 13 string evals) 16 42.3ms line • block • sub Template/Grammar.pm 5995 42.0ms line • block • sub /srv/otrs//Kernel/Config/Defaults.pm (including 1 string eval) 1932 41.4ms line • block • sub utf8_heavy.pl 2426 40.2ms line • block • sub /srv/otrs/Kernel/cpan-lib/JSON/PP.pm (including 20 string evals) 13010 36.0ms line • block • sub /srv/otrs/Kernel/cpan-lib/Apache2/Reload.pm 2205 31.2ms line • block • sub /srv/otrs//Kernel/System/Main.pm 163 29.3ms line • block • sub /srv/otrs//Kernel/System/DB.pm 6 28.4ms line • block • sub /srv/otrs//Kernel/System/Group.pm 203 21.2ms line • block • sub /srv/httpd/../modperl/lib/perl/5.14.2/ModPerl/RegistryCooker.pm (including 1 string eval) 6 20.7ms line • block • sub /srv/otrs//Kernel/System/LinkObject.pm 710 19.6ms line • block • sub /srv/otrs//Kernel/Language/en_CA_ITSMChangeManagement.pm 710 19.2ms line • block • sub /srv/otrs//Kernel/Language/en_GB_ITSMChangeManagement.pm 1820 18.1ms line • block • sub Template/Context.pm 56 16.2ms line • block • sub Cwd.pm (including 2 string evals) 348 15.5ms line • block • sub Template/Provider.pm 10 15.2ms line • block • sub /srv/otrs//Kernel/Output/HTML/Layout/Ticket.pm 36 15.0ms line • block • sub File/Find.pm 29 14.3ms line • block • sub Template/Parser.pm 1503 13.6ms line • block • sub Template/Filters.pm 748 13.3ms line • block • sub File/stat.pm 12 13.3ms line • block • sub /srv/otrs//Kernel/Output/HTML/Layout/LinkObject.pm 57 13.2ms line • block • sub /srv/otrs//Kernel/System/Web/InterfaceAgent.pm 23 13.0ms line • block • sub /srv/otrs//Kernel/System/User.pm 49 12.8ms line • block • sub POSIX.pm 16 11.9ms line • block • sub Template/Directive.pm 28 10.7ms line • block • sub /srv/otrs//Kernel/System/AuthSession/DB.pm 2306 10.5ms line • block • sub Class/Struct.pm (including 1 string eval) 63 10.4ms line • block • sub /srv/otrs//Kernel/System/Time.pm 543 10.4ms line • block • sub /srv/otrs//Kernel/System/Encode.pm 59 10.1ms line • block • sub /srv/otrs//Kernel/System/DB/mysql.pm 443 9.69ms line • block • sub /srv/otrs//Kernel/Language.pm
And another:
Profile of /srv/httpd/bin/httpd for 48.8s (of 51.6s), executing 1211808 statements and 324088 subroutine calls in 318 source files and 109 string evals. Top 15 Subroutines Calls P F Exclusive Time Inclusive Time Subroutine 27422 1 1 987ms 1.07s Encode::utf8::decode_xs (xsub) 11 1 1 672ms 882ms Template::Parser::_parse 2 1 1 640ms 1.71s Kernel::System::Fred::STDERRLog::CORE:readline (opcode) 228 19 12 633ms 1.05s Kernel::System::Main::Require 104 1 1 410ms 1.86s Kernel::System::ObjectManager::_ObjectBuild (recurses: max depth 1, inclusive time 267ms) 421 4 1 252ms 252ms Kernel::System::Main::CORE:ftis (opcode) 27 2 1 223ms 228ms Template::Document::new 100 1 1 186ms 186ms Kernel::System::Main::CORE:readline (opcode) 2 1 1 185ms 1.08s JavaScript::Minifier::minify 3 1 1 174ms 270ms Kernel::Config::Defaults::new 15 1 1 160ms 160ms Template::Provider::CORE:readline (opcode) 16352 2 1 159ms 249ms JavaScript::Minifier::action4 23 1 1 138ms 138ms DBI::db::do (xsub) 6 6 6 117ms 142ms utf8::SWASHNEW 15642 5 1 115ms 352ms JavaScript::Minifier::action3 See all 4654 subroutines
Source Code Files — ordered by exclusive time then name Stmts Exclusive Time Reports Source File 1 39.3s line • block • sub -e 50 1.79s line • block • sub /srv/otrs//Kernel/System/Fred/STDERRLog.pm 319261 1.09s line • block • sub /srv/otrs/Kernel/cpan-lib/JavaScript/Minifier.pm 450013 973ms line • block • sub Template/Parser.pm 9738 785ms line • block • sub /srv/otrs//Kernel/System/Main.pm 12143 345ms line • block • sub /srv/otrs//Kernel/System/DB.pm 84131 275ms line • block • sub /srv/otrs/Kernel/cpan-lib/CSS/Minifier.pm 3332 232ms line • block • sub Template/Document.pm (including 56 string evals) 9406 226ms line • block • sub /srv/otrs//Kernel/System/ObjectManager.pm 3121 193ms line • block • sub Template/Provider.pm 6996 170ms line • block • sub /srv/otrs/Kernel/Config/Files/ZZZAAuto.pm 23868 147ms line • block • sub /srv/otrs//Kernel/Config/Defaults.pm (including 1 string eval) 18762 143ms line • block • sub utf8_heavy.pl 33285 130ms line • block • sub Template/Context.pm 14866 130ms line • block • sub /srv/otrs//Kernel/Output/HTML/Layout.pm 21569 110ms line • block • sub Template/Directive.pm 14800 102ms line • block • sub /srv/otrs//Kernel/Output/Template/Document.pm 14016 93.1ms line • block • sub /Parser.yp 23534 82.7ms line • block • sub Template/Filters.pm 90 79.0ms line • block • sub /srv/otrs//Kernel/System/Ticket.pm 6676 77.3ms line • block • sub /srv/otrs//Kernel/Language/en_ITSMChangeManagement.pm 10857 71.4ms line • block • sub /srv/otrs/Kernel/cpan-lib/JSON/PP.pm (including 20 string evals) 27122 67.0ms line • block • sub /srv/otrs/Kernel/cpan-lib/Apache2/Reload.pm 5935 53.4ms line • block • sub /srv/otrs//Kernel/System/Ticket/TicketSearch.pm 1546 40.3ms line • block • sub /srv/otrs//Kernel/Output/Template/Provider.pm 42 40.1ms line • block • sub Template/Grammar.pm 11195 39.0ms line • block • sub /srv/otrs//Kernel/System/Encode.pm 4878 38.6ms line • block • sub /srv/otrs/Kernel/Output/HTML/Templates/Standard/AgentNavigationBar.tt 223 37.3ms line • block • sub /srv/otrs//Kernel/System/ITSMChange/ITSMWorkOrder.pm 881 35.0ms line • block • sub /srv/otrs//Kernel/System/ITSMChange.pm 1287 32.0ms line • block • sub File/Path.pm 34 32.0ms line • block • sub /srv/otrs//Kernel/Modules/AgentTimeAccountingEdit.pm 4096 31.0ms line • block • sub /srv/otrs//Kernel/System/Cache/FileStorable.pm 12 30.3ms line • block • sub /srv/otrs//Kernel/System/Ticket/Article.pm 5399 29.7ms line • block • sub Template/Stash.pm 314 29.2ms line • block • sub /srv/otrs//Kernel/System/Web/InterfaceAgent.pm 2737 28.0ms line • block • sub /srv/otrs//Kernel/System/DB/mysql.pm 439 27.8ms line • block • sub DBD/mysql.pm 1041 27.8ms line • block • sub /srv/otrs//Kernel/Output/HTML/TicketOverview/Small.pm 8 25.6ms line • block • sub /srv/otrs//Kernel/System/Ticket/TicketACL.pm 1331 25.2ms line • block • sub /srv/otrs//Kernel/System/User.pm 574 24.8ms line • block • sub /srv/otrs//Kernel/System/Group.pm 88 24.3ms line • block • sub /srv/otrs//Kernel/System/TimeAccounting.pm 1142 24.3ms line • block • sub /srv/otrs//Kernel/Modules/AgentTicketQueue.pm 2116 24.3ms line • block • sub /srv/otrs//Kernel/Language/en_CA_ITSMChangeManagement.pm 3186 23.8ms line • block • sub /srv/otrs//Kernel/System/Fred/ConfigLog.pm 2116 23.1ms line • block • sub /srv/otrs//Kernel/Language/en_GB_ITSMChangeManagement.pm 120 22.4ms line • block • sub File/Temp.pm 3003 22.4ms line • block • sub /srv/otrs//Kernel/Output/HTML/Layout/Template.pm 1260 22.3ms line • block • sub Storable.pm 1697 22.3ms line • block • sub /srv/otrs//Kernel/Output/Template/Plugin/OTRS.pm 284 22.2ms line • block • sub /srv/otrs//Kernel/System/DynamicField/Backend.pm 302 21.2ms line • block • sub /srv/otrs//Kernel/Output/HTML/Layout/Ticket.pm 54 20.9ms line • block • sub /srv/otrs//Kernel/System/Daemon/SchedulerDB.pm 2533 20.9ms line • block • sub /srv/otrs//Kernel/System/AuthSession/DB.pm 2969 20.3ms line • block • sub /srv/otrs//Kernel/Language.pm 26 19.9ms line • block • sub /srv/otrs/Kernel/cpan-lib/Date/Pcalc.pm 6 19.9ms line • block • sub /srv/otrs//Kernel/System/LinkObject.pm 2010 19.8ms line • block • sub /srv/otrs/Kernel/cpan-lib/Mail/Field.pm (including 3 string evals) 146 19.7ms line • block • sub /srv/otrs//Kernel/Output/HTML/FilterElementPost/CloseTicketView.pm 36 17.7ms line • block • sub /srv/otrs/Kernel/cpan-lib/MIME/Parser.pm 1286 17.3ms line • block • sub /srv/otrs//Kernel/System/Web/Request.pm 18 17.3ms line • block • sub /srv/otrs//Kernel/System/EmailParser.pm 36 17.2ms line • block • sub /srv/otrs//Kernel/System/ITSMConfigItem.pm 36 17.2ms line • block • sub /srv/otrs//Kernel/System/DynamicField/Driver/RemoteDB.pm 104 16.9ms line • block • sub /srv/otrs//Kernel/Output/HTML/FilterContent/Fred.pm (variables that impact regex performance for whole application seen here) 327 16.7ms line • block • sub /srv/httpd/../modperl/lib/perl/5.14.2/ModPerl/RegistryCooker.pm (including 1 string eval) 38 16.0ms line • block • sub /srv/otrs//Kernel/System/DynamicField/Driver/ITSMConfigItemReference.pm 1497 15.7ms line • block • sub File/stat.pm 2000 15.7ms line • block • sub /srv/otrs/Kernel/Output/HTML/Templates/Standard/AgentTicketOverviewNavBar.tt 36 15.3ms line • block • sub File/Find.pm 32 15.2ms line • block • sub /srv/otrs/Kernel/cpan-lib/MIME/Entity.pm 12 15.1ms line • block • sub /srv/otrs//Kernel/System/DynamicField/Driver/BaseDateTime.pm 2135 14.9ms line • block • sub /srv/otrs//Kernel/System/Cache.pm 55 14.6ms line • block • sub POSIX.pm 1622 14.6ms line • block • sub /srv/otrs/Kernel/Output/HTML/Templates/Standard/Header.tt 56 14.2ms line • block • sub Cwd.pm (including 2 string evals) 4446 14.2ms line • block • sub Class/Struct.pm (including 1 string eval) 347 14.0ms line • block • sub /srv/otrs//Kernel/System/DynamicField.pm 36 13.9ms line • block • sub /srv/otrs//Kernel/System/DynamicField/Driver/Date.pm 906 13.0ms line • block • sub /srv/otrs//Kernel/System/Loader.pm 978 12.8ms line • block • sub /srv/otrs//Kernel/System/Time.pm 2009 12.6ms line • block • sub File/Basename.pm 60 12.3ms line • block • sub /srv/otrs//Kernel/System/Queue.pm 1776 11.6ms line • block • sub /srv/otrs//Kernel/System/JSON.pm 24 11.6ms line • block • sub /srv/otrs//Kernel/System/DynamicField/Driver/Multiselect.pm 10 11.4ms line • block • sub /srv/otrs//Kernel/System/ITSMConfigItem/Version.pm 1034 10.6ms line • block • sub Template/Service.pm 12 10.6ms line • block • sub /srv/otrs//Kernel/Output/HTML/Layout/LinkObject.pm 24 10.4ms line • block • sub /srv/otrs//Kernel/System/Ticket/ArticleStorageDB.pm 6 10.1ms line • block • sub /srv/otrs//Kernel/Output/HTML/Layout/FAQ.pm 10 9.89ms line • block • sub /srv/otrs//Kernel/System/DynamicField/Driver/BaseSelect.pm
Filename -e Statements Executed 0 statements in 39.3s Subroutines Calls P F Exclusive Time Inclusive Time Subroutine 1 1 1 522µs 1.92ms Kernel::System::Fred::STDERRLog::BEGIN@82 1 1 1 13µs 13µs Kernel::System::CheckItem::BEGIN@282 Call graph for these subroutines as a Graphviz dot language file. Line State ments Time on line Calls Time in subs Code 0 23 9.46s Profile data that couldn't be associated with a specific line: # spent 9.39s making 3 calls to ModPerl::Registry::handler, avg 3.13s/call # spent 67.0ms making 8 calls to Apache2::Reload::handler, avg 8.38ms/call # spent 1.92ms making 1 call to Kernel::System::Fred::STDERRLog::BEGIN@82 # spent 516µs making 3 calls to Apache::DBI::__ANON__, avg 172µs/call # spent 366µs making 1 call to Apache::DBI::childexit # spent 165µs making 3 calls to CGI::_reset_globals, avg 55µs/call # spent 116µs making 1 call to File::Temp::END # spent 97µs making 1 call to DBI::END # spent 13µs making 1 call to Kernel::System::CheckItem::BEGIN@282 # spent 7µs making 1 call to APR::Pool::DESTROY 1 1 39.3s No source code available for non-file '-e'. You probably need to use a more recent version of perl. See savesrc option in documentation. 2 - - 82 # spent 1.92ms (522µs+1.39) within Kernel::System::Fred::STDERRLog::BEGIN@82 which was called: # once (522µs+1.39ms) by Kernel::System::Fred::STDERRLog::CORE:open at line 0
So what is -e file and executed 0 statements 39 seconds. Also ModPerl::Registry::handler takes 3 seconds per call. What could be the problem. If need more debug, please advise.

I just saw this thread. Did you try to figure out what parts of the pages were slow using Firebug or Chrome dev tools?

Hello. No, I didn't, cause page is sent as one. But I see who is slow by looking at round circle which indicates the page is loading. One direction is wait time, another direction is loading received data and showing it. And wait time is longer then receiving. On 08.07.2016 05:12, Ugo Bellavance wrote:
I just saw this thread. Did you try to figure out what parts of the pages were slow using Firebug or Chrome dev tools?
-- Mimiko desu.

You are not doing any server-side (Gzip) compression by chance? I saw minifying is done, which might increase the wait time, but perhaps that's not configurable. Sander

On 16-07-08 03:59 AM, Mimiko wrote:
Hello.
No, I didn't, cause page is sent as one. But I see who is slow by looking at round circle which indicates the page is loading. One direction is wait time, another direction is loading received data and showing it. And wait time is longer then receiving.
One page can be many components. On my OTRS system (3.3), the dashboard involves 10 HTTP requests. It would help you know if it is something trivial like the call to get OTRS news for example.

On 08.07.2016 23:36, Ugo Bellavance wrote:
One page can be many components. On my OTRS system (3.3), the dashboard involves 10 HTTP requests. It would help you know if it is something trivial like the call to get OTRS news for example.
This is what takes a lot of time: index.pl?Action=AgentCustomerInformationC Where Action can be any action. Perl execution is very slow.
participants (5)
-
Mimiko
-
Ralf Hildebrandt
-
Sander Goudswaard
-
Ugo Bellavance
-
Vogl, Yves