Hardening a Public Honeypot Server
I set out to build a simple public demo.
The idea was straightforward. A small PHP app, a SQLite database, and something exposed to the internet so I could see what kind of traffic it would attract.
It did not take long.
Within minutes, the logs started filling up with scans, random requests, and payloads hitting endpoints that did not even exist. At that point, it becomes pretty obvious that anything publicly accessible is going to get attention, whether you are ready for it or not.
So instead of trying to build something that was completely locked down, I shifted the goal a bit. I assumed that at some point something would break or be abused, and focused more on making sure the system could handle that without falling over.
Here is how I approached it.
Starting with the OS
Before getting into Apache or PHP, I spent some time making sure the base system was in a good place.
It is easy to skip over this and jump right into application configuration, but if the underlying system is not stable, everything built on top of it is going to inherit those problems.
Out of the box, Linux is designed to be flexible and broadly compatible. For something sitting directly on the internet, I wanted to tighten that up a bit. I adjusted a few sysctl settings to drop malformed packets, enable reverse path filtering to help with spoofing, ignore ICMP broadcast requests, and enable SYN cookies.
I added these to a new file at /etc/sysctl.d/99-security.conf:
# TCP/IP Stack Hardening
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.log_martians = 1
None of this is particularly complex, but applying it (via sysctl --system) helps reduce the amount of junk traffic the system has to deal with.
I also locked things down with UFW using a default deny policy and only allowing ports 80, 443, and SSH. That keeps the exposure surface small and predictable.
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow ssh
sudo ufw enable
The bigger piece for me was making sure the system could maintain itself. I enabled unattended upgrades for security patches, set the system to reboot automatically in the early morning if required, and enabled Livepatch so kernel updates could be applied without downtime. I also configured needrestart (setting $nrconf{restart} = 'a'; in /etc/needrestart/needrestart.conf) to run automatically so updates would not stall waiting for interactive input.
At that point, I was comfortable letting the system run without needing regular manual intervention.
Putting some boundaries around Apache
This setup uses mod_php, which means PHP runs inside the Apache process. If there is a vulnerability in the application, whatever code gets executed is running with Apache’s permissions.
Rather than reworking the stack, I decided to limit what Apache could actually do.
Systemd has built-in sandboxing features that make this fairly straightforward. By editing the Apache service (sudo systemctl edit apache2.service), I was able to make most of the filesystem read-only, hide user home directories, isolate temporary directories, and prevent privilege escalation.
[Service]
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
NoNewPrivileges=yes
As expected, this broke a few things.
Logs stopped writing at first, which was easy enough to trace. The database directory also needed write access. The more subtle issue was PHP sessions. Without access to the session directory, everything appeared to work, but sessions were not actually being saved, which caused issues with things like CSRF protection.
After explicitly allowing write access only where it was needed by appending ReadWritePaths to the systemd override, everything started behaving normally again.
# Explicitly grant write access ONLY to required directories
ReadWritePaths=/var/log/apache2
ReadWritePaths=/run/apache2
ReadWritePaths=/var/www/honeypot/data
ReadWritePaths=/var/lib/php/sessions
At that point, Apache was still functional, but operating within a much tighter set of boundaries.
Dealing with routing edge cases
Running a static site behind Cloudflare alongside a directly exposed honeypot introduced a couple of edge cases that were not immediately obvious.
The first showed up in the logs fairly quickly. Bots hitting the server by IP address instead of a hostname.
My initial instinct was to drop those requests with a catch-all VirtualHost that returned a 403. That works fine if your goal is to reduce noise, but in this case, it was doing the opposite of what I wanted. Those IP-based hits are part of the story. They are often the first touchpoints from scanners and bots, and there is value in seeing what they are trying to do.
So I backed that change out and let those requests fall through to the honeypot instead. I simply ensured the honeypot’s configuration file loaded first alphabetically (e.g., 000-honeypot.conf). If Apache doesn’t recognize the domain or is hit directly by an IP, it defaults to the honeypot. It adds noise to the data, but it also gives a more complete picture of what is happening.
The second issue was more subtle.
Because the honeypot exposes the server’s real IP address, it is possible to send a request with a forged Host header and reach the static site directly, bypassing Cloudflare entirely. That effectively skips the WAF.
To address that, I restricted access to the static site so that it only accepts requests coming from Cloudflare’s IP ranges. Inside the static site’s VirtualHost configuration, I replaced Require all granted with Cloudflare’s specific blocks (https://www.cloudflare.com/ips):
<Directory /var/www/static-site>
Options -Indexes
AllowOverride All
# Lock down to Cloudflare IPs to prevent WAF bypass
<RequireAny>
Require ip 103.21.244.0/22
Require ip 104.16.0.0/13
# ... (full Cloudflare IPv4/IPv6 list) ...
</RequireAny>
</Directory>
If a request does not come through Cloudflare, it does not get served.
Letting it take the hit
The usual approach with a public-facing system is to start blocking anything that looks suspicious.
For a honeypot, that works against what you are trying to do. Blocking traffic too early means you lose visibility into what is actually being sent. Instead, I focused on making sure the system could handle a high volume of requests without running out of resources.
One of the key changes was limiting the number of Apache worker processes. By editing the mpm_prefork.conf and setting a cap on MaxRequestWorkers, the server will queue requests under heavy load instead of spawning additional processes and exhausting memory.
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxRequestWorkers 150
MaxConnectionsPerChild 100
</IfModule>
This does not stop the traffic, but it keeps the system from collapsing under it.
Logging was another area that needed attention. With constant scanning and automated traffic, log files grow quickly. I adjusted log rotation (/etc/logrotate.d/apache2) to a “survival mode” setup. I configured it to keep only a few days of logs and to force a rotation once files reach a certain size.
/var/log/apache2/*.log {
daily
missingok
rotate 3
maxsize 250M
compress
...
This prevents the disk from filling up while still retaining enough data to be useful.
The SQLite database also needed a bit of maintenance. Since the data is largely noise, there is no need to keep it indefinitely. I set up a cron job that rebuilds the database every hour. To ensure zero downtime for the application, the bash script builds the new database in a temporary file and swaps it into place atomically.
# Build the new database in a temporary file
sqlite3 "$TMP_DB" < "$SCHEMA"
sqlite3 "$TMP_DB" < "$SEED"
# Set permissions
chown www-data:www-data "$TMP_DB"
chmod 664 "$TMP_DB"
# Atomically swap the temp database into production
mv "$TMP_DB" "$DB_FILE"
Because the new database is prepared ahead of time, the swap happens instantly without interrupting the application.
Locking down the static side
Even though the marketing site is just static content, it still shares the same Apache instance as the honeypot, so it is worth treating it carefully.
I made sure all files are owned by root and only readable by Apache. That removes the ability for the web process to modify anything in that directory.
sudo chown -R root:root /var/www/static-site
sudo find /var/www/static-site -type d -exec chmod 755 {} \;
sudo find /var/www/static-site -type f -exec chmod 644 {} \;
I also disabled PHP execution entirely within the static site configuration. If a PHP file somehow ends up there, it will be served as plain text rather than executed. On top of that, I added standard security headers such as HSTS, a basic content security policy, and protections against framing and content type sniffing.
Inside the Apache VirtualHost for the static site, it looks like this:
# Disable PHP Execution
php_admin_flag engine Off
# Defense in Depth Security Headers
Header always set X-Frame-Options "DENY"
Header always set X-Content-Type-Options "nosniff"
Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
Header always set Content-Security-Policy "upgrade-insecure-requests"
These are fairly standard, but still worth including to ensure the static assets remain isolated.
Where this ended up
This is not a system that is locked down in the traditional sense.
It is intentionally exposed and expected to be interacted with in ways you would not normally allow.
What it does do well is handle that interaction without falling apart. It limits its own resource usage, keeps itself updated, cleans up after itself, and continues to collect useful data.
The goal was never to keep everything out. It was to make sure that when something gets in, it does not take the rest of the system with it.