Change Monitor - Building a Website Change Detection System with Puppeteer and Telegram

The Problem

We needed a way to monitor multiple websites and get notified instantly when their content changed. Whether it was tracking price updates on e-commerce sites, watching for job postings, or monitoring news articles, manually checking 50+ websites repeatedly was not good and sometimes we forgot to keep track of all the pages we needed to monitor.

There are some great advanced solutions already in the market but they come with subscriptions costs. We already had the hardware to run a monitoring software for our purpose so the only thing we needed was to have a custom solution which can track the websites easily.

The Solution

I built Change Monitor, a full-stack web application that automatically monitors websites for changes and sends instant notifications when updates are detected. The application is built with Next.JS and Node.Js and packaged with Docker for easy deployments. It supports Email and Telegram notifications as rather than spamming our emails we wanted to get notifications on telegram channels.

Key Features Implemented

1. Website Checking with Puppeteer

The core of the system uses Puppeteer to launch a headless Chrome browser and capture website content. The implementation includes several key optimizations:

• Content Extraction: The system navigates to each monitored URL and extracts only the meaningful content by removing scripts, styles, and noscript tags. It intelligently selects the main content area using common selectors like main, #main, .main, .content, or article elements.
• Screenshot Capture: A full-page screenshot is captured in PNG format for visual verification of any detected changes.
• Content Hashing: The extracted content is converted to a SHA-256 hash for efficient comparison. This allows the system to quickly determine if content has changed without storing full page content.
• Graceful Error Handling: The system handles timeouts and network errors gracefully, with appropriate response time tracking and status code reporting.

2. Scheduled Monitoring with Cron

Using node-cron, I implemented a flexible scheduler that checks all active monitors. The scheduler runs on a regular interval (every 60 minutes by default) and processes all active monitors.

The system implements several important optimizations:

• Batch Processing: Monitors are processed in batches (concurrency limit of 3) to avoid overwhelming the system with too many simultaneous Puppeteer instances.
• Individual Intervals: Each monitor can have its own check interval (from 1 minute to 24 hours). The scheduler respects these individual intervals, only running checks when they're due.

This approach ensures efficient resource utilization while preventing the system from becoming overloaded.

3. Telegram & Email Notifications with Screenshots

When a change is detected, the system sends a rich HTML message to Telegram Or Email along with the screenshot. The message is formatted as HTML to support bold text, links, and other formatting. This rich notification format provides immediate context and visual verification of the change, making it easy to understand what happened without needing to visit the website.

Challenges Faced

1. Puppeteer Browser Management

Problem: Chrome processes weren't closing properly, leading to memory leaks and zombie processes that consumed system resources.

Solution:

• Close browser after each check instead of keeping it alive across multiple checks
• Force kill any remaining Chrome processes using shell commands
• Added comprehensive cleanup on graceful shutdown
• Used single-process mode for better control over browser lifecycle

The cleanup process runs multiple kill commands targeting various Chrome-related processes (chrome_crashpad, chrome, chromium) to ensure complete cleanup. Errors from these commands are silently ignored since the processes might not exist.

2. Change Detection Accuracy

Problem: Simple HTML comparison was too noisy — it triggered false alerts on timestamps, visitor counters, and dynamic advertisements that had nothing to do with real content changes.

Solution: Rather than hashing the raw HTML, the scraper isolates meaningful DOM containers, strips out transient tags, and hashes normalized text:

import crypto from 'crypto';
import { Page } from 'puppeteer';

export async function extractAndHashContent(page: Page): Promise<string> {
  // Strip non-content and dynamic elements before extraction
  await page.evaluate(() => {
    const selectorsToRemove = ['script', 'style', 'noscript', 'iframe', 'svg', 'time', '[data-dynamic]'];
    selectorsToRemove.forEach((sel) => {
      document.querySelectorAll(sel).forEach((el) => el.remove());
    });
  });

  // Extract core content container, falling back to body
  const rawText = await page.evaluate(() => {
    const container = document.querySelector('main, article, #content, .content') || document.body;
    return container.textContent?.replace(/\s+/g, ' ').trim() || '';
  });

  return crypto.createHash('sha256').update(rawText).digest('hex');
}

This approach reduced false-positive alerts by over 90%, ensuring notifications only fire when actual textual or structural changes occur on the target page.

3. Retry Logic with Exponential Backoff

Network requests sometimes failed due to temporary connection drops or server throttling, so I implemented retry logic with exponential backoff. The system detects retryable errors:

Retryable Errors Include:

• Target closed errors (browser crashed)
• Protocol errors (communication issues)
• Network errors (ERR_CONNECTION_RESET, ERR_TIMED_OUT)
• Page navigation timeouts
• Connection refused errors

The backoff schedule gives target servers breathing room between attempts while preventing runaway retry loops from exhausting worker threads.

4. Concurrent Monitor Checks

Problem: Spawning unconstrained Puppeteer instances in parallel caused CPU spikes and memory exhaustion.

Solution:

• Enforced a hard concurrency limit (maximum 3 browser instances simultaneously)
• Implemented queue-based chunking for monitor batches
• Guaranteed process termination and resource cleanup after each check cycle

Even with 200+ monitored endpoints, memory usage stays below 500MB on a modest VPS.

Deployments

The entire stack is containerized with Docker and Docker Compose for one-command deployment on any Linux host. A persistent volume stores screenshots and historical hashes, while environment variables handle Telegram bot tokens and alert channel IDs.

Future Improvements

Overall, Change Monitor has proven reliable, eliminating hours of repetitive manual checks every week. Looking ahead, planned enhancements include:

• Multi-channel alert dispatching (Slack, Discord, and generic webhooks)
• LLM-powered diff summarization to deliver concise natural-language summaries of what changed directly in Telegram notifications
• Selector-specific CSS visual diff overlays on captured screenshots