Sending emails, generating reports, importing data, cleaning up old records: almost every application has tasks that run in the background (background tasks) without making users wait.

On your own computer, a thread or asyncio.create_task is enough for this. On web hosting, however, things do not work quite as you might expect. In this article, you will learn why and which solutions work.

Table of Contents

• Why Threads Do Not Work in the Application
• Choose the Right Solution
• Cron: Tasks at Scheduled Times
  • Pattern: Delayed Processing
• PM2: A Continuously Running Process
  • Automatic Startup After a Server Restart
  • What to Keep in Mind
• Frequently Asked Questions

Why Threads Do Not Work in the Application

A Python application created in Setup Python App does not run continuously:

• the web server starts it when a request arrives and distributes the work across multiple worker processes,
• between requests, processes are suspended, so threads and background tasks do not run,
• after a few minutes without visitors, the application shuts down completely, along with any unfinished tasks.

Moreover, a thread started when the application is imported can cause requests to hang. Therefore, do not start threads, schedulers (such as APScheduler) or event loops during import in files imported by passenger_wsgi.py.

FastAPI BackgroundTasks also run, but the user waits for a response until the task has finished.

Choose the Right Solution

Cron: Tasks at Scheduled Times

Set up a cron job in cPanel → Cron Jobs or from the terminal using crontab -e. On web hosting, an individual task can run no more than once every 30 minutes.

Example: Run the report.py script in the application’s virtual environment every night at 3:15 a.m.:

15 3 * * * export OPENBLAS_NUM_THREADS=1; cd $HOME/myapi && $HOME/virtualenv/myapi/3.11/bin/python report.py >> $HOME/myapi/cron.log 2>&1

It is important to use Python from the application’s virtual environment; otherwise, the script cannot find the installed packages.

Pattern: Delayed Processing

When a user triggers a longer task (such as importing a file), the application saves the task to a database and immediately responds with “task is being processed”. Every 30 minutes, cron runs a script that processes all pending tasks and marks them as completed.

*/30 * * * * cd $HOME/myapi && $HOME/virtualenv/myapi/3.11/bin/python process_tasks.py >> $HOME/myapi/tasks.log 2>&1

PM2: A Continuously Running Process

If you need a process that runs continuously (such as uvicorn for WebSocket or a worker that processes a task queue as new tasks arrive), use the PM2 process manager. PM2 automatically restarts a process if it fails.

Installation:

export PATH=/opt/alt/alt-nodejs22/root/usr/bin:$PATH
mkdir -p ~/pm2 && cd ~/pm2
npm install pm2

Starting a Python script:

export PATH=/opt/alt/alt-nodejs22/root/usr/bin:$PATH
export PM2_HOME=~/pm2/.pm2
~/pm2/node_modules/.bin/pm2 start ~/myapi/worker.py \
  --name worker \
  --interpreter ~/virtualenv/myapi/3.11/bin/python \
  --cwd ~/myapi
~/pm2/node_modules/.bin/pm2 save

Useful commands (with the same two export lines as above):

~/pm2/node_modules/.bin/pm2 list          # list processes
~/pm2/node_modules/.bin/pm2 logs worker   # logs
~/pm2/node_modules/.bin/pm2 restart worker
~/pm2/node_modules/.bin/pm2 delete worker

Automatic Startup After a Server Restart

Processes stop during server maintenance. To have PM2 start them again, create the ~/pm2/startup.sh script:

#!/bin/bash
export PATH=/opt/alt/alt-nodejs22/root/usr/bin:$PATH
export PM2_HOME=$HOME/pm2/.pm2
export OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1
PM2=$HOME/pm2/node_modules/.bin/pm2
# restore the saved state (pm2 save) if no processes are running
$PM2 jlist | grep -q '"status":"online"' || $PM2 resurrect

Then add two cron jobs:

@reboot /bin/bash $HOME/pm2/startup.sh >/dev/null 2>&1
*/30 * * * * /bin/bash $HOME/pm2/startup.sh >/dev/null 2>&1

The first job starts the processes when the server boots, while the second checks every 30 minutes whether they are running and restarts them if necessary. Remember to run pm2 save whenever you change your processes.

What to Keep in Mind

When running background tasks, keep the following limitations and recommendations in mind:

• A continuously running process consumes memory from your hosting plan at all times (the Green plan provides 1 GB) and counts towards the process limit. For continuously running processes, we recommend Turbo Hosting, which offers more memory and CPU cores.
• A worker should not keep the CPU busy in a loop. It should wait between checks (time.sleep) and process tasks as they arrive.
• Under the Terms of Use, web hosting is not intended for applications that continuously consume system resources (data processing and collection, analytics). For such applications, we recommend a VPS Server.

Frequently Asked Questions

Find answers to frequently asked questions about running background tasks.

Can I Use Celery, RQ or Redis?

Celery and RQ require a message broker (Redis or RabbitMQ) and continuously running workers. For smaller applications, a database-backed task queue and cron are a simpler and more reliable solution. For larger systems, we recommend VPS Hosting.

Will Uvicorn or a Script Started with nohup Stop When I Close SSH?

No. A process started with nohup ... & keeps running. However, if the process crashes or the server restarts, it won’t start again automatically. That’s why we recommend using PM2.

How Can I Tell Whether Cron Is Working?

Redirect the output to a file (>> cron.log 2>&1) or enter an email address in cPanel → Cron Jobs to receive the output from each execution.

If you need help setting up background tasks, we are here to help.