503 Error¶

503 errors are often caused by a malfunction in PHP, and are unrelated to the web server. A common reason for a 503 error is a PHP crash. In this section we will go over basic steps to troubleshoot 503 errors, common causes of 503 errors, and a few examples that show the steps in practice.

If your server is running on a control panel, such as cPanel or Plesk, and you run into a 503 error, you should temporarily switch to Apache, and attempt to reproduce the error.

• If you do not experience the same issue with Apache, then the 503 error is a LiteSpeed issue. Please log a ticket with us, so we may take a a look.
• If you do experience the same issue with Apache (as you will most of the time), then your 503 error is not a web server issue and is not appropriate for a ticket. Nonetheless, we have provided this documentation to give you some troubleshooting tips. Please run through the steps outlined below until you find the issue.

$ sudo touch /usr/local/lsws/admin/tmp/.lsphp_restart.txt
$ sudo systemctl restart lsws

Log Files¶

There are many different log files that may help you determine the cause of the 503 error. In this section we will discuss the different log files, their locations, and what information they provide.

When troubleshooting 503 errors it is ideal to check these logs in the following order:

1. PHP Error Log
2. Web Server Error Log
3. Standard Error Log
4. System Log

PHP Error Log¶

This error log is generated by PHP when it detects an error, warning, etc., depending on your log level in PHP. It is usually generated inside of the same folder that the PHP script runs in.

Fatal error: Call to undefined function my_function() in /home/mysite/public_html/test.php on line 2

These types of errors are usually found inside of the PHP application you are trying to run. It is best to forward these logs to the developer of the PHP application.

Web Server Error Log¶

This error log is generated by the web server. It can usually be found in either the /usr/local/lsws/logs/ or /usr/local/apache2/logs/ directory. The web server error log may provide helpful hints whether the web server caused PHP to fail.

[INFO] [319934] [1.1.1.1:49873-1#APVH:lsapi] connection to [uds://tmp/lshttpd/APVH-php56.sock] on request #0, confirmed, 0, associated process: 0, running: 0, error: Too many open files!

Standard Error Log¶

This error log is generated by standard errors generated while PHP is running. It can usually be found in either the /usr/local/lsws/logs/ or /usr/local/apache2/logs/ directory. This log is usually the most helpful when determining why PHP is throwing 503s.

[STDERR] fork() failed, please increase process limit: Cannot allocate memory

System Log¶

This error log is generated by the operating system. It can usually be found as either /var/log/messages or /var/log/syslog. This log is helpful if PHP is being affected by the operating system.

lfd[18304]: *User Processing* PID:18264 Kill:1 User:xxxxx VM:538(MB) EXE:/usr/local/lsws/fcgi-bin/lsphp-5.4.42 CMD:lsphp5

Troubleshooting¶

Basic¶

Display Errors¶

In a production environment it is common to disable PHP application errors from displaying in order to secure the server in the event of a issue. This however is counter productive when trying to diagnose 503 errors.

If you are experiencing 503 errors for a specific site/domain, try adding the following right after the opening <?php tags, to the same script with the 503 error:

<?php # The opening PHP tag
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);

When you visit the web page now, it should show an error like this:

Fatal error: Call to undefined function my_function() in /home/mysite/public_html/test.php on line 2

Normally this has nothing to do with the web server and is actually an issue with the code. We suggest checking Stack Overflow or contacting the developer of the software for a fix.

PHP Info Page¶

The PHP info page gives us a lot of detail about your PHP installation. This page is a very basic file that should not throw a 503 error.

<?php
echo phpinfo();
?>

Here is a list of some of the useful information the PHP info page gives us:

• Extensions
• Memory Limit
• Execution Time
• Other Settings

With this information, you can see if OPCache is enabled, what the memory limit is, which extensions are loaded, etc; all useful information when looking for the cause of a 503 error.

This page can also be used to compare differences between Apache's and LiteSpeed's PHP implementations.

OPCache¶

OPCache is the process of storing bytecode in shared memory so PHP does not need to process the same calls over and over, but it can occasionally cause issues. When troubleshooting 503 errors, it is best to have OPCache disabled.

Common OPCache Extensions:

• OPCache
• APC/u
• XCache
• eAccelerator

To disable the OPCache extensions follow the extensions guide.

[STDERR] zend_mm_heap corrupted

Extensions¶

These are a couple of PHP extensions that have been known to cause issues, especially when they are outdated:

• ZendGuardLoader
• Suhosin
• ionCube
• Imunify360

We recommended continually checking to see if there are any updates for these extensions. If you are already using the latest versions, try to disable the extensions by following the Disable guide .

If you disable these plugins and the 503 error is still being shown, disable all other extensions that were not enabled by default, by following the Disable guide.

Disk Space¶

PHP will sometimes store files on your server in order to function properly. If your disks become full, especially the partition that houses /tmp/, it could be the cause of your 503 error.

To check your disk space usage you can run $ sudo df -h in a terminal.

$ sudo df -h
Filesystem                          Size  Used Avail Use% Mounted on
devtmpfs                            901M     0  901M   0% /dev
tmpfs                               915M   96K  915M   1% /dev/shm
tmpfs                               915M   89M  827M  10% /run
tmpfs                               915M     0  915M   0% /sys/fs/cgroup
/dev/mapper/cl_centos                29G  4.2G   25G  15% /
/dev/vda1                           976M  184M  726M  21% /boot
tmpfs                               183M     0  183M   0% /run/user/0

If you have full disks, clean up unused and old files. This should solve the 503 error.

Memory Limit¶

When PHP reaches its allocated memory limit, it will crash. There are 2 different memory limits that PHP must adhere to: its internal memory limit, set by php.ini, and LiteSpeed's memory limit, set inside of the web server for when the PHP process spawns.

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 12864 bytes)

[STDERR] fork() failed, please increase process limit: Cannot allocate memory

PHP Memory¶

If you get a memory error like the one above, the first step in troubleshooting it is to increase the memory limit. Adjust the following setting inside of php.ini:

memory_limit = 100M

To figure out where your php.ini file is located, and find out your current memory limit, you can create a PHP info page.

<?php
echo phpinfo();
?>

Sometimes scripts may append lines to .htaccess to override the PHP settings. This can override what is set in your php.ini file.

Here is an example of a memory_limit entry inside of .htaccess:

php_value memory_limit xxxM

You can add/delete/modify this line inside of your .htaccess file to suit your needs.

CloudLinux Memory Limit¶

If you are running CloudLinux and Lightweight Virtual Environment (LVE) it is possible that you are hitting the limits set by that account.

Here shows accounts that have hit a limit, depending on what limit the account has hit you should adjust the limit. To adjust the limits follow CloudLinux's guide.

Process Memory¶

If you have increased the memory limit in php.ini and confirmed you are not hitting a LVE limit then it is possible LSPHP needs to increase its virtual memory.

1. Go to your Web Admin console ( https://YOUR_SERVER_IP:7080 ) and log in.
2. Next, navigate to Configuration > Server > PHP.
3. Now click Edit for PHP Handler Defaults.
4. Change the following settings, then click Save.
   • Memory Soft Limit: 4097M
   • Memory Hard Limit: 4098M
5. Now that everything has been set up, navigate to Actions and press the icons next to Restart Detached PHP Processes. Then Apply Changes / Graceful Restart.

Max Execution Time¶

If you have a max execution time that is not high enough to handle long running requests then you may get the following error in your web server error log: error: Connection reset by peer! and have a value of 0, meaning that the process never finished the request. These errors will often be explained in your stderr.log. If the stderr.log does not provide any insight then you can try to increase the max execution time by following the steps in the Control guide.

CSF/LFD¶

If you have CSF/LFD installed they may be causing the 503 error. These programs may be killing the process before it finishes. CSF/LFD has two settings that can kill a LSPHP process: PT_USERTIME and PT_FORKBOMB.

Kill¶

If a LSPHP process takes too long to complete it is possible that the PT_USERTIME feature will be called to kill the process. If it does kill the process you may receive an email or an entry in /var/log/lfd.log with the following message:

lfd[18304]: *User Processing* PID:18264 Kill:1 User:xxxxx VM:538(MB) EXE:/usr/local/lsws/fcgi-bin/lsphp-5.4.42 CMD:lsphp5

This means that we need to whitelist the LSPHP process from CSF/LFD to prvent this from happening.

To do this, run the following commands in a terminal:

$ sudo echo "pexe:/usr/local/lsws/fcgi-bin/lsphp.*" >> /etc/csf/csf.pignore
$ sudo csf -r; sudo service lfd restart

Forkbomb¶

In rare cases, LSPHP will be killed by PT_FORKBOMB for having too many processes spawned at any given time. This will, in return, end up sending the signal 9/SIGKILL to the main Litespeed process. To stop csf/lfd from killing the LSPHP and LiteSpeed processes, the following settings inside of /etc/csf/csf.conf should be adjusted:

PT_LIMIT - Default: 10
PT_USERPROC - Default: 10
PT_FORKBOMB - Default: 0
PT_INTERVAL - Default 60

These settings should be adjusted on a per case basis as each scenario is different. They should be increased slightly higher than the current values until the issue stops occurring. For the changes to take effect, both csf and lfd need to be restarted. This can either be done via the WHM plugin or through a shell terminal by using the following command:

$ sudo csf -r; sudo service lfd restart

Extension Mismatch¶

Running LSPHP extensions that are not built for the version of LSPHP you are using may cause a 503 error. If this is the case, you will see the following messagin inside of a stderr.log:

Warning: PHP Startup: imap: Unable to initialize module
Module compiled with module API=20090626
PHP    compiled with module API=20100525
These options need to match in Unknown on line 0

To fix this, you must rebuild the module and/or PHP, making sure to use a version of PHP that works with the module. Also, make sure that the correct extension path is used.

Extension Loading Order¶

With PHP, some extensions require other extensions be loaded first in order to work efficently. This is accomplished by changing the extensions's .ini file from mysql.so to 20-mysql.so. The 20- value is the loading priority. It can be changed from 01- to 99-, with lower values loading before the higher values. You can find more information on the PHP site.

If this is the cause of the issue, then you should see the following entry in your stderr.log.

PHP Warning:  PHP Startup: Unable to load dynamic library

Open Files Limit¶

If you see the following entry inside of your web server error long, then your system is maxed out of the amount of files it can open.

[INFO] [319934] [1.1.1.1:49873-1#APVH:lsapi] connection to [uds://tmp/lshttpd/APVH-php56.sock] on request #0, confirmed, 0, associated process: 0, running: 0, error: Too many open files!

To check your current limit, run the following command:

$ sudo ulimit -n

It will show you your system's current limit. To fix this issue, increase the limit by running:

$ sudo ulimit -n <Increased_Number>

Now you should no longer see a 503 error because LSPHP can open more files on your system.

Advanced¶

The comprehensive guide above lists the most likely reasons you'd receive a 503 error, but in some circumstances, it may be difficult to locate the root cause. If PHP crashes, you can enable a core dump and use GDB to debug the core file. Even if you find an internal PHP bug causing the crash, there are limited actions you can take. You can disable the module or change to another PHP version without the bug. Reporting the bug to PHP is an option, but a solution may not be provided in a timely manner.

If there is a core file generated and you've tried all of the above methods without finding a solution, try to strace the PHP process for a hint. Just be prepared that the strace log may be extremely long and overwhelming. You likely need some knowledge and experience to understand what the log entries mean; it can even be a challenge locating the relevant sections of the log.

Strace¶

Use the following command to strace the PHP process to find out what happened:

strace -tt -T -f -p <pid>

This command will show you what PHP is doing and where the issues is.

[INFO] [1.1.1.1:48506] connection to [/tmp/lshttpd/APVH_vhost_Suphp.sock] on request #0, confirmed, 1, associated process: 791228, running: 1, error: Connection reset by peer!

strace -tt -T -f -p 791228

while true; do if mypid=`ps aux | grep $USERNAME | grep lsphp | grep $SCRIPTNAME | grep -v grep | awk '{print $2; }' | tail -1`; then strace -tt -T -f -p $mypid; fi ; done

Core Files¶

If you find that PHP has crashed (as demonstrated by the process being killed by signal other than value 15), a core dump will allow you to look further into the cause of the crash. These core files are usually generated inside of the folder that PHP crashed with.

Enable Core Files¶

This section assumes you have core dumps enabled for centOS or Ubuntu already.

1. Go to your Web Admin console ( https://YOUR_SERVER_IP:7080 ) and log in.
2. Next, navigate to Configuration > Server > PHP.
3. Now click Edit for PHP Handler Defaults.
4. Change the following settings then click Save.
   • Environment: LSAPI_ALLOW_CORE_DUMP=1
5. Now that everything has been set up, navigate to Actions and press the icons next to Restart Detached PHP Processes. Then Apply Changes / Graceful Restart.

Analayze Core Files¶

GNU Debugger (GDB) uses the syntax gdb <path/to/lsphp/binary> <path/to/core/file>. The <path/to/lsphp/binary> is the path to the PHP binary that the PHP application was using. The path to the core file is the location of the core file.

First to load GDB you would run `

Then you will see a prompt with (gdb) then you will want to type bt then hit enter.

This will generate output like the following:

gdb <path/to/lsphp/binary> <path/to/core/file>
Program received signal SIGSEGV, Segmentation fault.
0x000000000061c91b in ?? ()
(gdb) bt
#0 0x000000000061c91b in ?? ()
#1 0x0000000000641ac3 in zend_stack_push ()
#2 0x000000000060bbf9 in ?? ()
#3 0x0000000000611515 in lex_scan ()
#4 0x000000000061fb60 in ?? ()
#5 0x0000000000608223 in ?? ()
#6 0x0000000000614965 in compile_file ()
#7 0x00007fd3c99eda21 in ?? () from /opt/alt/php55/usr/lib64/php/modules/phar.so
#8 0x00007fd3d0993359 in ?? () from /opt/alt/php55/usr/lib64/php/modules/opcache.so
#9 0x00007fd3d0994187 in ?? () from /opt/alt/php55/usr/lib64/php/modules/opcache.so
#10 0x00000000006b54da in ?? ()
#11 0x00000000006b68e8 in execute_ex ()
#12 0x000000000064239c in zend_execute_scripts ()
#13 0x00000000005e2fb0 in php_execute_script ()
#14 0x00000000006f29ff in ?? ()
#15 0x00000000006f2c5c in ?? ()
#16 0x00000000006f2f85 in ?? ()
#17 0x00007fd3d37cdd1d in __libc_start_main () from /lib64/libc.so.6
#18 0x0000000000424bc9 in _start ()

Here we can see that the some of the modules that loaded when it crashed were phar.so and opcache.so. To try to troubleshoot this, you would want to disable phar.so and opcache.so to figure out which module is causing the issue.

(gdb) frame 13
(gdb) print (char *)(executor_globals.function_state_ptr->function)->common.function_name
(gdb) print (char *)executor_globals.active_op_array->function_name
(gdb) print (char *)executor_globals.active_op_array->filename

Auto Fix 503¶

In LiteSpeed Web Server's WebAdmin Console, there is a feature to Auto Fix 503 Error. It specifies whether to try to fix the 503 Service Unavailable error by restarting the server gracefully. A 503 error is usually caused by malfunctioning external applications and a web server restart can often fix the error temporarily. If Auto Fix 503 Error is set to Yes, the server will restart automatically whenever there are more than thirty 503 errors within a 30 second span. This feature is enabled by default, but you can disable it by navigating to LSWS Admin > Server > General and setting Auto Fix 503 Error to No.

Auto Fix 503 Error is not a permanent fix. You should not rely on this feature to fix recurring 503 errors. Going through the above steps to find a permanent fix is always recommended.