A Monitoring Alert Is Not Automatically a Server Outage

Server monitoring is designed to reliably determine whether a service is reachable. For this purpose, Livewatch.de checks, for example, HTTP and HTTPS services, ping, DNS, ports, and HTTP performance scripts at short intervals.

But what happens if it's not the monitored server that's down, but the monitoring request itself is blocked?

This can happen faster than many administrators realize. A firewall, a web application firewall (WAF), Fail2Ban, a security system, a reverse proxy, or a manually configured access restriction can automatically or intentionally reject a monitoring request.

For monitoring purposes, this initially appears as a classic case of unavailability: The expected response is not received, or an error code is returned.

Therefore, it's crucial to distinguish between an actual outage and a blocked monitoring request.

Why Can a Monitoring Request Be Blocked?

Modern servers and networks have numerous security mechanisms. Under certain circumstances, legitimate monitoring requests may be classified as unwanted or suspicious.

Typical causes include:

• Firewall rules
• Web application firewalls
• Automatic IP blocking
• Fail2Ban or similar protection mechanisms
• Rate limiting
• DDoS protection
• Geo-blocking
• Upstream reverse proxies
• Access restrictions to specific IP addresses
• Manually maintained blacklists
• Security software on the web server
• Protection mechanisms against bots and automated access

It becomes particularly problematic when a security system blocks the monitoring server's IP address after several automated requests.

The monitored service may then continue to function perfectly normally. Only Livewatch.de is no longer permitted to query it.

Therefore, Livewatch.de does not monitor from just one location.

To better detect such situations, Livewatch.de performs checks from multiple locations.

This has a crucial advantage:

If, for example, a request from location A is blocked, but the service from location B remains accessible, it strongly suggests that the actual service is not down.

In this case, a general server outage is rather unlikely. Instead, the IP address of location A, for example, might have been blocked.

What happens if all tests are blocked?

The situation becomes even more difficult if all test locations are unable to establish a connection.

Then Livewatch.de cannot definitively determine why there is no response.

The server has actually failed.

Or:

A firewall or security software is blocking the monitoring requests.

Or:

A network disruption is preventing access.

For Livewatch.de, these situations initially appear similar.

This is an important technical limitation of any external monitoring system: A monitoring system can detect that a check is failing. However, the exact cause is not always within the monitoring provider's control.

What should customers check when an alert is triggered?

Therefore, if Livewatch.de reports an outage, you shouldn't immediately assume that your own server has actually failed.

With a few simple checks, the cause can often be quickly narrowed down.

1. Test the service yourself

The first step is the easiest:

Access the monitored service yourself.

For example, for a website:

https://www.example.de

Does the website work from your computer?

If so, this doesn't automatically mean that Livewatch.de is issuing a false alarm. It could be that your IP address is allowed, but the IP address of the monitoring server is blocked.

This is a very common misconception.

2. Test from a different internet connection

Testing from a different internet connection is even more informative.

For example:

• Smartphone using mobile data
• Different internet connection
• Computer on a different network
• External server

If the website works at your own workstation but not over a different connection, this indicates a network, routing, or firewall problem.

If, however, it works everywhere while Livewatch.de continues to report an error, you should examine your security and firewall configuration more closely.

--

3. Check the HTTP status code

When monitoring HTTP and HTTPS, it's worth taking a look at the actual HTTP status code.

Examples:

• 200 – Request successful
• 301 / 302 – Redirected
• 403 – Access denied
• 404 – Resource not found
• 429 – Too many requests
• 500 – Internal server error
• 502 / 503 / 504 – Problems with server, proxy, or backend

In particular, 403 and 429 can indicate that a security component is rejecting the automated request.

4. Check Firewall and WAF

If the service is accessible to regular users, but Livewatch.de reports an outage, the next step should be to check the firewall or WAF.

Of particular interest are rules that react to the following criteria:

• Source IP address
• Number of requests
• User agent
• Request frequency
• Countries
• Specific URL paths
• Bot detection
• Failed login attempts
• Unusual request behavior

Pay particular attention to the logs at the time of the Livewatch alert.

These often contain an indication that the connection reached the server but was subsequently rejected by a security component.

--

5. Check Automatic IP Blocking

Programs like Fail2Ban or similar systems can automatically block IP addresses.

This can happen, for example, if a security mechanism classifies a certain number of requests within a specific time period as suspicious.

The consequence:

Livewatch.de regularly queries the server → Security software detects the requests → IP address is blocked → Monitoring reports unavailability.

The server itself may be fully functional.

Therefore, if an alert is triggered, the following should be checked:

• Which IP addresses were blocked?
• When was the block issued?
• Which rule triggered the block?
• Was a monitoring IP address automatically blocked?
• Is there a whitelist for monitoring systems?

--

6. Check DNS

When monitoring DNS, it should also be verified whether DNS resolution is actually working.

For example, the following question is relevant:

Does the DNS server deliver the same response from different locations?

A DNS problem can cause a domain to be accessible to some users but not to others.

A comparison between different networks can also be helpful here.

--

7. Ping is not the same as HTTP

Another common misconception concerns ping tests.

A server can fail to respond to a ping and still provide websites, APIs, or other services without any problems.

For example, a firewall might block ICMP packets while TCP connections on ports 80 or 443 work perfectly.

Therefore:

A failed ping does not automatically mean the server is down.

Conversely, a successful ping does not necessarily mean the website is working.

That's why it's crucial to investigate the specific service that triggered the alert.

8. Reviewing Server and Web Server Logs

In the event of an actual problem, the logs are often the most important source of information.

Depending on the software used, the following logs, among others, should be checked:

• Web server logs
• Firewall logs
• WAF logs
• Reverse proxy logs
• Application logs
• System logs
• DNS logs
• Fail2Ban logs

The time of the alarm is crucial.

If Livewatch.de, for example, reports an outage at 2:32 PM, you shouldn't just look at the current status. Specifically search the logs for events around 2:32 PM.

A simple troubleshooting process

The following sequence can help with a Livewatch alarm:

1. Access the service yourself

→ Is the website or service working?

2. Test from a different network

→ Does the service work over a mobile network, for example?

3. Check HTTP status or error message

→ 403, 429, 500, 502, or 503?

4. Check firewall and WAF

→ Was a request rejected?

5. Check IP blocking

→ Was a monitoring IP blocked?

6. Check server and security logs

→ Are there any entries relevant to the time of the alarm?

7. Check DNS and routing

→ Is the service reachable from other networks?

8. Check monitoring configuration

→ Is a URL, port, or protocol being monitored that has been intentionally restricted?

Monitoring shouldn't become a problem itself

The task of a monitoring system is to detect problems as early as possible. At the same time, it must be considered that a monitoring system itself is part of the communication chain.

Between Livewatch.de and the monitored service, there may be numerous components:

Livewatch monitoring server → Internet → Firewall → WAF → Reverse proxy → Web server → Application

Each of these components can influence or block a request.

If one of these components rejects the request, Livewatch.de can initially only determine:

"The expected response did not arrive."

However, the cause is not necessarily the monitored server.

Conclusion

A monitoring alert should always be taken seriously. However, it should not automatically be equated with an actual server outage.

Especially with modern security solutions, it can happen that automated monitoring requests are blocked. Livewatch.de therefore tries to better pinpoint such situations by conducting checks from different locations.

However, if all checks simultaneously receive no response, even Livewatch.de cannot reliably distinguish whether the monitored service has actually failed or whether the monitoring requests are being blocked, for example, by a firewall, WAF, or automatic IP blocking.

Therefore, it is particularly important for the operator to review their own firewall, WAF, and server logs.

Knowing the IP addresses of the Livewatch monitoring servers and taking them into account in their security components, or whitelisting them, also prevents legitimate monitoring requests from being mistakenly treated as attacks or unwanted automated access.

This is how Livewatch.de and the local server and security logs complement each other perfectly: Livewatch.de detects external unavailability – the customer's logs then help to identify the actual cause.