Complete Guide to Debug LOAD Issue in LAMP Stack [Linux/Apache/Mysql/PHP]

General • PHP October 16, 2021
load_debug_flow
Debug LOAD Issue in LAMP Stack
  1. Problem Statement
  2. Root Cause Analysis [RCA]

Problem Statement

There is php, apache , mysql based application, the stack is hosted on Linux.

This is a single server application, where PHP and apache are on an ubuntu-based Linux server and MySQL is on another server.

let’s suppose the application becomes unresponsive after a certain load. Now you want to RCA and optimize the bottleneck of the problem, the following will provide you a complete guide to debug LOAD issue in LAMP stack

LAMP Stack Component

To figure out the load issue, first, we need to figure out the dependent items of the stack, from where the problem may arise. We divide this into two parts, hardware, and software

Hardware

  1. Linux server hosting apache2
  2. Linux server hosting MySQL

Software

  1. Apache/Nginx webserver
  2. PHP processing [running as module/fpm/fastcgi]
  3. Mysql server

Root Cause Analysis [RCA]

Flow Of Analysis

Considering the situation where the site is becoming unresponsive,

the very first thing that is coming to consideration is, Hardware limitation

Monitor Hardware Component

Q1:- are the resources enough to handle so many requests?

To get the answer to this question, we analyze the system state. considering that we are on Linux OS [ubuntu], however, most *nix are same.

— check for web server machine

  • If possible, put monitoring on CPU utilization. You can use htop/top command to check the process and load
  • Put monitoring on ram utilization. htop/top command can be used to check the log
  • In case of the machine become unresponsive, check for System logs [syslogs], mostly under var/log/syslog location
    • tail -f /var/log/syslog
  • Check the stale process using
    • strace or pmap <pid>
    • ls /proc/<pid>
    • cat /proc/<pid>/status
    • cat /proc/cpuinfo
    • cat /proc/meminfo
    • cat /proc/zoneinfo
    • cat /proc/mounts

htop/top is only helpful if you monitor during load as they shows current state of the system otherwise put the monitoring by using any tool.

There are various tool like cloud instances also provide resource monitoring as Cloudwatch in AWS, or, one can setup open source tool like nagios, zabbix, opennms or there are handy utilitites like, glance, atop, etc https://www.cyberciti.biz/tips/top-linux-monitoring-tools.html

There are 2 main factors to analyze

Apart from these 2, one can also check for another two factors. As most of the cloud systems provide a good bandwidth allocation and that can also be checked from Network monitoring and Most systems are equipped with SSD, so these are not the major factory but can play a vital role if your application highly depends on that, like you are using firebase data to read and write which need higher I/O or delivering large media file

— check for the database server machine

If the database server is deployed on another machine, then, we need to follow the same steps as we covered in the earlier section to figure if the database server is enough or not.

In case, you are using AWS RDS service for the database, you can use AWS cloudwatch or RDS monitoring tool with CPU matrix or connection matrix to figure out the root cause.


Most of the cloud provider provides these type of system/instance/machine monitoring tool, where we check the hardware component dependency and bottleneck areas.

Now there may be two situations,

  1. Either server resources are NOT enough,
  2. Or server resources are available even at peak load.

Server resources are NOT enough

ACTION

* Scale Up / Scale Out the resources [Vertical Scaling / Horizontal Scaling]

* Validate the request are legtimate, means there is no bocus request flooding your server to keep busy

If hardware is the bottleneck, we need to upgrade the resource either by following Vertical Scaling or Horizontal Scaling

Server resources are enough

If we are at this point, where resources are available like, servers are healthy and under the threshold limit, it means that hardware is not the bottleneck, we should check for the software level bottleneck.

Monitor Software Component

Q2: if the server resources are enough, then what might be the root cause of the unresponsive web app?

In a typical PHP website, the software stack includes.

Tagged with: • • • •

Leave a Reply

Your email address will not be published. Required fields are marked *