Skip to main content

Puppeteer Blocked by Anti-Bot? From Chrome CDP to Electron Alternative

· 7 min read

Encountered this issue while building a data collection tool for a client. Here's the full troubleshooting journey from Puppeteer to Electron.

TL;DR​

Puppeteer stealth plugin cannot bypass advanced CAPTCHA-based anti-bot systems. Switching to Chrome CDP remote debugging led to WSL2 network isolation and Chrome single-instance issues. The final solution: use Electron BrowserWindow to load the target site, let the user log in manually, and automatically extract cookies via session.cookies.get() — eliminating anti-bot detection and cross-platform problems entirely.

Scenario 1: Puppeteer Blocked by Anti-Bot​

Problem​

Using puppeteer-extra + puppeteer-extra-plugin-stealth for automated login, the browser triggers CAPTCHA interception immediately after launch. Even after passing the CAPTCHA, subsequent pages detect the automation environment and force an exit.

import puppeteer from 'puppeteer-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';

puppeteer.use(StealthPlugin());

const browser = await puppeteer.launch({
headless: false,
args: [
'--disable-blink-features=AutomationControlled',
'--no-sandbox',
'--disable-infobars',
],
});

const page = await browser.newPage();
await page.goto('https://target-site.com/login');
// Anti-bot CAPTCHA system detects automation, page is blocked

Root Cause​

Advanced anti-bot CAPTCHA systems don't just check basic fingerprints like navigator.webdriver. They detect automation through multiple dimensions: Chromium build signatures, Canvas/WebGL rendering differences, mouse trajectory patterns, and even DevTools Protocol call stacks. The stealth plugin fixes known fingerprint leaks but cannot eliminate the fundamental differences between Puppeteer's Chromium and a real Chrome browser.

After 8 rounds of debugging (removing timeouts, listening for disconnect events, investigating environment differences, fully configuring stealth plugin, etc.), we confirmed that no configuration could bypass the detection.

Solution​

Abandoned Puppeteer automated login entirely. Switched to Chrome DevTools Protocol (CDP) to connect to the user's real browser and extract cookies.

Scenario 2: WSL2 Cannot Connect to Windows Chrome CDP​

Problem​

Running a Node.js script in WSL2, connecting to Windows Chrome's debugging port via chrome-remote-interface results in a connection timeout:

import CDP from 'chrome-remote-interface';

const client = await CDP({
host: 'localhost',
port: 9222,
});
// Error: connect ECONNREFUSED 127.0.0.1:9222

Chrome launched on Windows with:

chrome.exe --remote-debugging-port=9222

Running curl localhost:9222/json in Windows PowerShell works fine, but WSL2 cannot connect.

Root Cause​

Chrome's --remote-debugging-port=9222 binds to 127.0.0.1 by default — the Windows loopback address. WSL2 and Windows have separate network stacks. localhost inside WSL2 points to the Linux loopback address, not Windows. So accessing localhost:9222 from WSL2 actually hits Linux's port 9222, not Windows Chrome.

Solution​

Add --remote-debugging-address=0.0.0.0 when launching Chrome to make CDP listen on all network interfaces:

chrome.exe --remote-debugging-port=9222 --remote-debugging-address=0.0.0.0

Or configure port forwarding using Windows netsh:

netsh interface portproxy add v4tov4 listenport=9222 listenaddress=0.0.0.0 connectport=9222 connectaddress=127.0.0.1

Important

--remote-debugging-address=0.0.0.0 exposes the CDP port to the local network, which is a security risk. Only use this in internal development environments. For production, always combine with firewall rules to restrict access.

Scenario 3: Chrome Ignores --remote-debugging-port​

Problem​

Chrome is already running. Launching Chrome again with --remote-debugging-port=9222 silently ignores the flag. Chrome simply opens a new tab in the existing window, and the CDP port is not opened:

# Chrome already running
chrome.exe --remote-debugging-port=9222
# No error, but port 9222 is not listening

Root Cause​

Chrome is designed as a single-instance application. When it detects an existing Chrome process, the newly launched Chrome forwards the URL (and other command-line arguments) to the existing process and exits. Flags like --remote-debugging-port only take effect when the process is first created — the existing process never dynamically loads these parameters.

Solution​

Close all Chrome processes first, then relaunch with the flag:

# Windows
taskkill /F /IM chrome.exe
chrome.exe --remote-debugging-port=9222

# macOS
pkill -f "Google Chrome"
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222

You can also use --user-data-dir to specify a separate profile directory, avoiding conflicts with your daily Chrome:

chrome.exe --remote-debugging-port=9222 --user-data-dir="C:\chrome-debug-profile"

The three scenarios above show that relying on an external Chrome process for cookie extraction is too fragile under WSL2 + anti-bot detection + process management constraints. The final solution: replace external Chrome with Electron's BrowserWindow.

Why Electron Works​

  1. No anti-bot detection: Electron's built-in Chromium shares the same rendering engine as Chrome. Anti-bot systems do not flag it as an automated environment.
  2. No external Chrome dependency: No need to manage Chrome processes, CDP ports, or profile directories.
  3. No WSL2 networking issues: Electron runs natively on the target OS — no cross-OS network isolation problems.
  4. No single-instance conflicts: Electron creates its own BrowserWindow — no conflict with the user's daily Chrome.

Complete Implementation​

Open login window and poll for cookies:

import { BrowserWindow } from 'electron';

let cookieWindow = null;

function openLoginWindow() {
cookieWindow = new BrowserWindow({
width: 1000,
height: 700,
title: 'Login to Target Platform',
webPreferences: {
// Key: use isolated session, separate from main window
partition: 'cookie-login',
contextIsolation: true,
nodeIntegration: false,
},
});

cookieWindow.loadURL('https://target-site.com/login');

// Poll for target cookie
const interval = setInterval(async () => {
const cookies = await cookieWindow.webContents.session.cookies.get({
domain: '.target-site.com',
});

const sessionCookie = cookies.find((c) => c.name === 'session_token');
if (sessionCookie) {
clearInterval(interval);

// Build full cookie string
const cookieStr = cookies
.map((c) => c.name + '=' + c.value)
.join('; ');

// Write to environment variable
process.env.SESSION_COOKIE = cookieStr;
updateEnvFile('SESSION_COOKIE', cookieStr);

cookieWindow.close();
}
}, 2000);

cookieWindow.on('closed', () => {
clearInterval(interval);
cookieWindow = null;
});
}

Preload script exposing IPC interface:

import { contextBridge, ipcRenderer } from 'electron';

contextBridge.exposeInMainWorld('electronAPI', {
extractCookie: () => ipcRenderer.invoke('extract-cookie'),
onCookieExtracted: (callback) =>
ipcRenderer.on('cookie-extracted', (_, data) => callback(data)),
});

Renderer process usage:

// Detect Electron environment
if (window.electronAPI) {
document.getElementById('extractBtn').addEventListener('click', () => {
window.electronAPI.extractCookie();
});

window.electronAPI.onCookieExtracted((data) => {
console.log('Cookie extracted:', data);
});
}

Important

  • partition: 'cookie-login' creates an isolated session. The login window's cookies won't pollute the main window. If you need to share login state, remove the partition option or use the same partition name.
  • A 2-second polling interval balances UX and performance. Don't replace setInterval with a while + await loop — that would block the renderer process.
  • session.cookies.get() only returns cookies from the current session. You cannot read cookies across partitions.

FAQ​

What to do when Puppeteer stealth plugin is still detected by anti-bot?​

The stealth plugin only bypasses basic fingerprint checks (like navigator.webdriver). Advanced anti-bot systems detect automation through browser behavior and low-level API characteristics, including Chromium build signatures, Canvas rendering differences, and mouse trajectory patterns. The stealth plugin cannot fully cover all detection vectors. Use Electron BrowserWindow to load the target site and extract cookies via session.cookies.get() after the user logs in manually.

Why is Chrome --remote-debugging-port not working?​

Chrome uses a single-process architecture. When a Chrome process is already running, the --remote-debugging-port flag is silently ignored. The newly launched Chrome simply opens a new tab in the existing instance without enabling the debugging port. You need to close all Chrome processes first using taskkill /F /IM chrome.exe (Windows) or pkill -f "Google Chrome" (macOS), then relaunch with the flag. Alternatively, use --user-data-dir to specify a separate profile directory to avoid conflicts.

How to connect to Chrome debugging port from WSL2?​

Chrome's --remote-debugging-port binds to Windows 127.0.0.1 by default, but WSL2 has its own network stack where localhost points to the Linux loopback address. Two solutions: first, add --remote-debugging-address=0.0.0.0 when launching Chrome to make CDP listen on all interfaces; second, configure port forwarding on Windows using netsh interface portproxy to forward WSL2 requests to the Windows CDP port.


Need help with data collection or automation tools?

Contact Us

Vercel Serverless Function Multi-Level Route 404? Bypass the Catch-All Trap with Rewrites

· 5 min read

Encountered these issues while deploying a Hono backend to Vercel Serverless Functions. Three cascading problems: esbuild bundling format errors, native dependency resolution failures, and the most subtle one — catch-all routes silently failing on nested paths.

TL;DR​

  1. esbuild + Hono: Use --format=cjs, exclude native deps with --external
  2. Multi-level route 404: api/[[...path]].ts is unreliable. Use api/index.ts + vercel.json rewrite instead
  3. Better Auth client: baseURL requires a full URL, use window.location.origin to build it

Issue 1: Vercel's Built-in TS Compilation Fails​

api/[[...path]].ts directly imports Hono's app.ts. Vercel compiles it with built-in TypeScript 5.9.3 (nodenext mode), which throws:

Relative import paths need explicit file extensions
Cannot find name 'process'
Module '"@libsql/client"' declares 'Client' locally, but it is not exported

The root cause is Vercel's built-in TS compiler enforcing strict nodenext module resolution, which is incompatible with how Hono and Turso client libraries export their types.

Solution: Pre-bundle with esbuild and have Vercel execute the compiled output directly.

esbuild server/src/app.ts \
--bundle --platform=node --format=cjs \
--outfile=dist/_server.cjs \
--external:@libsql/client

api/[[...path]].ts becomes a one-liner:

import app from '../../dist/_server.cjs';
export default app;

Why CJS Instead of ESM?​

Bundling with --format=esm causes dotenv's internal require("path") to throw Dynamic require is not supported. This is an ESM spec limitation that CJS doesn't have.

If you've also run into module resolution issues with Node.js ESM dynamic import, the root cause is the same — ESM enforces strict module format rules while CJS is more lenient.

Native Dependency Handling​

@libsql/client includes native binaries (@libsql/linux-x64-gnu). After esbuild bundling, the runtime can't find the module. Solution:

  1. Exclude with --external:@libsql/client
  2. Declare @libsql/client as a dependency in root package.json so Vercel installs it to node_modules

Issue 2: Nested Routes Intercepted by Vercel​

After fixing esbuild bundling, single-level routes like /api/health and /api/tasks worked fine. But all nested paths (e.g., /api/auth/sign-in/email) returned Vercel-level 404:

HTTP/2 404
x-vercel-error: NOT_FOUND
content-type: text/plain; charset=utf-8

Debugging Process​

By comparing response headers, the breakpoint was clear:

PathReaches Hono?Key Indicators
/api/health✅x-vercel-cache: MISS, CORS headers present
/api/gate✅CORS headers present
/api/gate/session❌x-vercel-error: NOT_FOUND, no CORS

Single-level paths reach the function, nested paths get intercepted. The issue is unrelated to path naming (paths without auth also 404) and unrelated to Vercel's internal routing rules. The root cause: api/[[...path]].ts catch-all pattern only matches single-level paths.

Solution​

Drop the catch-all. Use api/index.ts + vercel.json rewrite instead:

{
"rewrites": [
{ "source": "/api/(.*)", "destination": "/api/index" },
{ "source": "/((?!api/).*)", "destination": "/index.html" }
]
}

Rename api/[[...path]].ts to api/index.ts (content unchanged). All /api/* requests are forwarded to /api/index by the rewrite rule, and Hono handles internal routing.

Post-deploy verification:

curl -sI https://example.com/api/gate/session
# HTTP/2 404
# access-control-allow-credentials: true ← Reaches Hono
# x-vercel-cache: MISS ← No longer a Vercel-level 404

All three paths /api/gate, /api/gate/, /api/gate/session now reach Hono. POST requests also work:

curl -X POST https://example.com/api/gate/sign-in/email \
-H "Content-Type: application/json" \
-d '{"email":"[email protected]","password":"123456"}'
# {"message":"Invalid email or password","code":"INVALID_EMAIL_OR_PASSWORD"}

Auth routes are fully functional.

Important Notes

  • Rewrite rule order matters — /api/(.*) must come before the SPA fallback
  • After deployment, browsers may cache old JS files. If the frontend still requests old paths, use Ctrl+Shift+R to force refresh
  • If you're also troubleshooting stale deployments, check out Debugging Frontend Deploy Not Updating

Issue 3: Better Auth Client baseURL Error​

With routes working, the frontend threw:

BetterAuthError: Invalid base URL: /api/gate
Caused by: TypeError: Failed to construct 'URL': Invalid URL

Better Auth client internally uses new URL() to parse baseURL, which fails with relative paths.

Solution: Use window.location.origin to build a full URL:

export const authClient = createAuthClient({
baseURL: `${window.location.origin}/api/gate`,
})

Locally this expands to http://localhost:5173/api/gate (Vite proxy forwards it), and in production to https://your-domain.com/api/gate — no environment variables needed.

Complete Configuration Reference​

The three key files in their final working state:

vercel.json

{
"buildCommand": "pnpm run check && pnpm exec esbuild server/src/app.ts --bundle --platform=node --format=cjs --outfile=dist/_server.cjs --external:@libsql/client && pnpm -F client run build",
"outputDirectory": "client/dist",
"rewrites": [
{ "source": "/api/(.*)", "destination": "/api/index" },
{ "source": "/((?!api/).*)", "destination": "/index.html" }
]
}

api/index.ts

import app from '../dist/_server.cjs';
export default app;

client/src/lib/auth-client.ts

import { createAuthClient } from 'better-auth/react';

export const authClient = createAuthClient({
baseURL: `${window.location.origin}/api/gate`,
});

FAQ​

Why does Vercel api/[[...path]].ts not match multi-level paths?​

Vercel's catch-all pattern only matches single-level paths (/api/foo), not nested paths (/api/foo/bar). Use api/index.ts with a vercel.json rewrite rule to forward all /api/* requests explicitly.

How to bundle Hono with esbuild for Vercel Serverless?​

Bundle with esbuild in CJS format (--format=cjs), exclude native dependencies like @libsql/client with --external, and declare them in root package.json so Vercel installs them.

How to fix Better Auth client Invalid base URL error?​

createAuthClient's baseURL does not accept relative paths. Use window.location.origin to build a full URL that works in both local development and production.


Running into similar Vercel deployment issues? Get in touch and let's talk about your tech stack.

Get in Touch

Chrome Extension Messages Leaking? postMessage targetOrigin Wildcard Exposes Data to Third-Party Iframes

· 3 min read

Encountered this issue while developing a Chrome extension data collection tool for a client. Here's the root cause and solution.

TL;DR​

window.postMessage(data, '*') broadcasts the message to all frames in the page, including third-party iframes. If your Chrome extension passes user data (like memberId, business reports) via postMessage, any embedded iframe can intercept it. Change '*' to window.location.origin to restrict the recipient precisely.

The Problem​

A Chrome extension's injected script passes e-commerce data to its content script via postMessage:

// ❌ Unsafe: message broadcasts to ALL frames
window.postMessage({
type: 'CCL_SHOP_REPORT_DAILY',
memberId: 'b2b-2214126315258ad300', // user ID
rows: [{ uv: 403, payAmt: 19478.47 }] // business data
});
// Equivalent to window.postMessage(data, '*')

If the page embeds third-party iframes (ads, analytics, social widgets), their message event listeners will also receive this message.

Root Cause​

The second argument to postMessage, targetOrigin, determines the message's delivery scope:

targetOriginBehavior
'*' or omittedBroadcasts to all frames, no origin check
'https://example.com'Only delivers to frames with origin https://example.com
window.location.originOnly delivers to frames same-origin as the current page

When omitted, the browser defaults to '*'. This is especially dangerous in Chrome extension scenarios — injected scripts run on e-commerce platform pages that may contain multiple third-party iframes.

Solution​

Wrap postMessage in a Safe Helper​

// safePostMessage: enforce window.location.origin
function safePostMessage(data) {
window.postMessage(data, window.location.origin);
}

// Usage
safePostMessage({
type: 'CCL_SHOP_REPORT_DAILY',
subType: 'daily',
memberId: memberId,
rows: [row]
});

Validate Origin on the Receiving End Too​

// Content script message listener
window.addEventListener('message', (event) => {
// ✅ Verify origin
if (event.origin !== window.location.origin) return;

// ✅ Validate message structure
if (!event.data || typeof event.data.type !== 'string') return;

switch (event.data.type) {
case 'CCL_SHOP_REPORT_DAILY':
handleDailyReport(event.data);
break;
case 'CCL_ITEM_WEEKLY_REPORT':
handleWeeklyReport(event.data);
break;
}
});

When is '*' Acceptable?​

Only when the message contains zero sensitive information and the recipient's origin is unpredictable — e.g., a pure UI state notification like "panel opened". Even then, window.location.origin is safer.

Caveats​

Caveats

  • Chrome extension MAIN world scripts and content scripts run in separate JavaScript isolation contexts — postMessage is their standard communication channel. Protect it carefully (if you encounter duplicate message processing after hot reload, you'll also need to manually clean up old listeners).
  • Receiver-side event.origin validation and sender-side targetOrigin restriction are both required. One-sided protection is incomplete.
  • If messages need to cross origins (e.g., from page to extension background), use chrome.runtime.sendMessage (see Service Worker Token Sync) instead of postMessage.

UPSERT Writes All Zeros? Drizzle sql Template Pitfall with Parameterized Values vs SQL Expressions

· 3 min read

Encountered this issue while building an e-commerce analytics platform for a client. Here's the root cause and solution.

TL;DR​

In Drizzle ORM's sql template tag, sql.join(values.map(v => sql(v))) parameterizes all values. If the values array contains SQL expressions (like date_trunc('week', '2026-05-17'::date)::date), PostgreSQL treats them as plain strings and throws invalid input syntax for type date. SQL expressions must use sql.raw() or be written separately in the template.

The Problem​

Data collection pipeline: Chrome extension → CCLHub server → Analytics API → PostgreSQL. Symptoms:

  1. CCLHub logs show correct collected data (uv: 403, payAmt: 19478.47)
  2. Analytics API returns 200 success
  3. But database query shows all zeros: uv: 0, pay_amt: 0.00
-- Actual database data
report_date | uv | pay_amt | reveal_cnt
-------------+-----+----------+------------
2026-05-12 | 392 | 7333.67 | 11879 -- old data fine
2026-05-13 | 0 | 0.00 | 0 -- new data all zeros!

Analytics error log reveals:

PostgresError: invalid input syntax for type date:
"date_trunc('week', '2026-05-17'::date)::date"

Root Cause​

The original code mixed parameterized values with SQL expressions:

// ❌ Problem code
const insertVals: (string | number | null)[] = [
String(shop_id),
String(platform_id),
reportDate,
tenant_id,
`date_trunc('week', '${reportDate}'::date)::date`, // ← SQL expression
];

// sql.join parameterizes ALL values, including the date_trunc expression
await db.execute(sql`
INSERT INTO table (..., week_start_date)
VALUES (${sql.join(insertVals.map(v => sql`${v}`), sql`,`)})
...
`);

Generated SQL:

-- PostgreSQL receives $5 as a literal string value
INSERT INTO table (..., week_start_date)
VALUES ($1, $2, $3, $4, $5, ...)
-- $5 = "date_trunc('week', '2026-05-17'::date)::date" ← treated as string!

PostgreSQL tries to parse "date_trunc('week', '2026-05-17'::date)::date" as a date type → error.

Why zeros instead of an error? Because the same table has a separate inquiry INSERT (PARTIAL UPSERT) that succeeded, creating rows with dashboard columns defaulting to 0. The daily report UPSERT failed but didn't roll back the existing rows.

Solution​

Separate SQL expressions from parameterized values using sql.raw() or direct template embedding:

// ✅ Fix: separate parameterized values from SQL expressions
const insertCols = ['shop_id', 'platform_id', 'report_date', 'tenant_id'];
const insertVals: (string | number | null)[] = [
String(shop_id), String(platform_id), reportDate, tenant_id,
];

// 19 data columns parameterized normally
for (const [apiKey, dbCol] of Object.entries(DAILY_COLUMNS)) {
insertCols.push(dbCol);
insertVals.push(row[apiKey] != null ? String(row[apiKey]) : '0');
}

// week_start_date uses SQL expression, NOT in parameterized array
await db.execute(sql`
INSERT INTO table (${sql.raw(insertCols.join(', '))}, week_start_date)
VALUES (
${sql.join(insertVals.map(v => sql`${v}`), sql`,`)},
date_trunc('week', ${reportDate}::date)::date -- ← directly in template
)
...
`);

Key distinction:

ApproachHow Drizzle handles itWhat PostgreSQL receives
sql template interpolationParameterized ($N)String literal
sql.raw(expression)Inlined into SQLSQL expression
Direct in sql templatePart of templateSQL expression

Caveats​

Caveats

  • sql.raw() has SQL injection risk — never use it for user input. In this example, reportDate comes from an internal API with controlled format
  • Drizzle's sql template tag auto-parameterizes all interpolations — this is a safety feature, but SQL function calls shouldn't be parameterized
  • If the entire SQL is dynamically constructed, consider using Drizzle's query builder API instead of raw SQL
  • Database connection config has its own pitfalls — if you're connecting to the wrong PostgreSQL instance, Docker might be silently occupying the port
  • Environment variable loading order is another common trap — JWT signing silently failing is a classic example of dotenv running after the import chain

JWT Signing Silently Fails? Check Your Node.js Environment Variable Loading Order

· 3 min read

Encountered this issue while building a SaaS authentication system for a client. Here's the root cause and solution.

TL;DR​

In Node.js ES Modules, import statements execute before dotenv.config(). If module-level code reads process.env.JWT_SECRET, it gets undefined, causing JWT signing to use the string "undefined" as the secret — no errors thrown, but all token verification fails. The fix: lazy initialization.

The Problem​

JWT login returns 200, but all subsequent requests return 401. Investigation reveals:

  1. Tokens generated at login can't be verified by jwtVerify()
  2. Every server restart invalidates all previously issued tokens
  3. console.log(process.env.JWT_SECRET) outputs undefined
// jwt.ts — module-level code
import crypto from 'crypto';

// ❌ This line executes BEFORE dotenv.config(), JWT_SECRET is undefined
const SECRET = crypto.createSecretKey(
new TextEncoder().encode(process.env.JWT_SECRET)
);

The worst part: no error is thrown. new TextEncoder().encode(undefined) encodes the string "undefined" into bytes, producing a valid but wrong secret key.

Root Cause​

ES Module import statements are statically hoisted:

// server.ts (entry file)
import { router } from './routes/auth'; // ← runs first
import { authenticateToken } from './middleware/auth'; // ← runs first

dotenv.config(); // ← runs AFTER all imported modules execute

Execution order:

  1. Node.js scans all import statements and builds the dependency graph
  2. Executes all imported modules' top-level code depth-first (jwt.ts's const SECRET = ... runs here)
  3. Returns to server.ts, runs dotenv.config()
  4. Now .env is loaded into process.env

So jwt.ts module-level code reads process.env.JWT_SECRET as undefined.

Solution​

Move secret initialization into a function — env var is read on first call, not at import time:

import crypto from 'crypto';

let _secret: crypto.KeyObject | null = null;

function getSecret(): crypto.KeyObject {
if (!_secret) {
const secretValue = process.env.JWT_SECRET;
if (!secretValue) {
throw new Error('JWT_SECRET environment variable not set');
}
_secret = crypto.createSecretKey(
new TextEncoder().encode(secretValue)
);
}
return _secret;
}

// Use getSecret() everywhere the key is needed
export async function generateToken(payload: any): Promise<string> {
return new SignJWT(payload)
.setProtectedHeader({ alg: 'HS256' })
.sign(getSecret()); // ← deferred until runtime
}

No dependency on entry file import order — safe regardless of when called.

Option 2: Call dotenv at the Very Top of Entry File​

// server.ts — ensure these lines come before ALL imports
import 'dotenv/config'; // or require('dotenv').config()
import express from 'express';
// ...other imports

Limitation: If another entry point (cron job, worker) forgets this line, the bug resurfaces.

Caveats​

Caveats

  • This pitfall affects all module-level env var reads, not just JWT — database connections, API keys, etc.; ESM module resolution has another common gotcha — missing .js extensions in dynamic imports causes module-not-found errors in production
  • require('dotenv').config() only guarantees order in CommonJS; ES Module import always executes before runtime code
  • Lazy initialization works well for: secrets, DB connection pools, external API clients, and other one-time resources; if you're using the jose library with Node 24, also watch out for KeyObject format changes

Chrome Extension Collecting Empty Data? Same-Site Cookies with Different Formats Break ID Mapping

· 3 min read

Encountered this issue while building an e-commerce data collection system for a client. Here's the root cause and solution.

TL;DR​

The 1688 platform sets two cookies (last_mid and unb) both containing user IDs, but in different formats (b2b-xxx vs plain digits). The database stores the b2b- prefix format. The original code iterated the cookie array in the outer loop, matching unb first — the strict equality check failed, and all collected data was written with zero values because it couldn't map to a shop.

Fix: Put the cookie key priority list in the outer loop and the cookie array in the inner loop, ensuring high-priority keys are checked first.

Problem​

After collecting daily shop reports from 1688, all dashboard metrics in the database were 0, while inquiry data had values:

shop_id | report_date | reveal_cnt | uv | pay_amt | effective_inq_users
2 | 2026-05-13 | 0 | 0 | 0.00 | 56
2 | 2026-05-14 | 0 | 0 | 0.00 | 41

Server log: No shop found for memberId: 2214126315258

But the database stored platform_account_id as b2b-2214126315258ad300.

Root Cause​

Two cookies with different formats on the same domain​

unb=2214126315258                          # plain digits
last_mid=b2b-2214126315258ad300 # b2b- prefix + suffix

The shops.platform_account_id column stores the b2b- prefix format. The code used strict equality:

const match = mapping.find(m => m.platform_account_id === memberId);
// "2214126315258" !== "b2b-2214126315258ad300" → no match

Iteration order trap​

The original code had the cookie array in the outer loop and the key list in the inner loop:

// ❌ Wrong: cookie array outer, key priority ineffective
var keys = ['last_mid', '__last_memberid__', 'unb'];
for (var i = 0; i < cookies.length; i++) { // outer: cookies
var pair = cookies[i].trim();
for (var k = 0; k < keys.length; k++) { // inner: keys
if (pair.indexOf(keys[k] + '=') === 0) {
return pair.substring(keys[k].length + 1);
}
}
}

document.cookie order is not guaranteed. If unb appears before last_mid in the array, it gets matched first — returning the plain-digit format and making last_mid priority useless.

Solution​

Swap loop levels: key priority list in the outer loop, cookie array in the inner loop:

// ✅ Correct: key priority list in outer loop
var keys = ['last_mid', '__last_memberid__', 'unb'];
for (var k = 0; k < keys.length; k++) { // outer: iterate keys by priority
for (var i = 0; i < cookies.length; i++) { // inner: search all cookies
var pair = cookies[i].trim();
if (pair.indexOf(keys[k] + '=') === 0) {
return pair.substring(keys[k].length + 1);
}
}
}

The key list is iterated by priority in the outer loop, so last_mid is always checked first regardless of document.cookie order, guaranteeing the user ID format matches the database.

Verification​

// Confirm both cookies exist in browser console
document.cookie.split(';')
.filter(c => /last_mid|unb/.test(c.trim()))
.map(c => c.trim())
// ['unb=2214126315258', 'last_mid=b2b-2214126315258ad300']

Note

This issue doesn't affect chrome.cookies.get() in extension pages — it queries by name directly. But when parsing document.cookie strings, always pay attention to loop levels. Also, if your extension passes data via postMessage, watch out for the targetOrigin wildcard security risk; and if messages are processed twice after hot reload, you'll need to manually manage listener lifecycle.


Node.js ESM Dynamic Import Can't Find Module? Check the File Extension

· 3 min read

Encountered this issue while building a SaaS analytics platform for a client. Here's the root cause and solution.

TL;DR​

In Node.js ESM mode, import('./path/to/module') doesn't auto-resolve ./path/to/module.js. If the TypeScript build output is missing .js extensions, the module throws ERR_MODULE_NOT_FOUND. When this import is inside deferred logic (timers, conditionals), the app starts fine but crashes later — PM2 shows a climbing restart count.

Fix: Ensure all ESM dynamic imports include .js extensions, and automate this in the build pipeline.

Problem​

PM2 showed the app restarting continuously:

│ name             │ ↺    │ status │ uptime │
│ analytics-api │ 9 │ online │ 28m │

Error log repeated every few minutes:

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/app/dist/domains/video/cleanup'
imported from /app/dist/server.js

But the file clearly exists:

$ ls dist/domains/video/
cleanup.js executor.js queue.js

Root Cause​

ESM doesn't auto-resolve file extensions​

Node.js CommonJS (require()) automatically tries .js, .json, and other extensions. ESM (import) does not.

// ❌ ESM can't find the module
import('./domains/video/cleanup')
// Node.js looks for: ./domains/video/cleanup (exact path, no extension)
// Actual file: ./domains/video/cleanup.js

// ✅ Must include .js extension
import('./domains/video/cleanup.js')

Deferred imports hide the problem​

The import was inside a deferred execution:

// server.ts — doesn't execute immediately on startup
import('./domains/video/cleanup.js').then(({ startCleanupScheduler }) => {
startCleanupScheduler(); // triggers seconds later
});

The app starts successfully (DB connection, port binding all fine). When the timer fires, the import fails → process crashes → PM2 restarts → starts fine again → timer fires again → crashes again. This creates a crash loop.

Why did it work before?​

Previous deployment used a build script that included a post-build step to fix import paths. One deployment skipped this step and deployed raw tsc output — tsc doesn't modify import paths in output files.

Solution​

1. Write .js extensions in TypeScript source​

TypeScript officially recommends writing .js extensions even in .ts files:

// ✅ Write .js even in .ts source files
import('./domains/video/cleanup.js').then(({ startCleanupScheduler }) => {
startCleanupScheduler();
});

2. Automated post-build fix (recommended)​

Add an import-fixing script to the build pipeline:

{
"scripts": {
"build": "tsc && node fix-imports.js"
}
}

Core logic of fix-imports.js:

import { readFileSync, writeFileSync, readdirSync } from 'fs';
import { join } from 'path';

function fixImports(dir) {
for (const file of readdirSync(dir, { withFileTypes: true })) {
const fullPath = join(dir, file.name);
if (file.isDirectory()) {
fixImports(fullPath);
} else if (file.name.endsWith('.js')) {
let content = readFileSync(fullPath, 'utf8');
// Fix dynamic imports
const fixed = content.replace(
/import\(['"](\.[^'"]+)['"]\)/g,
(match, path) => path.endsWith('.js') ? match : match.replace(path, path + '.js')
);
// Fix static imports
const fixed2 = fixed.replace(
/from\s+['"](\.[^'"]+)['"]/g,
(match, path) => path.endsWith('.js') ? match : match.replace(path, path + '.js')
);
if (fixed2 !== content) {
writeFileSync(fullPath, fixed2);
}
}
}
}

The build pipeline automatically appends .js to all relative import paths, keeping TypeScript source extension-free while preventing module-not-found errors in ESM deployments.

Note

  • This only affects Node.js ESM mode ("type": "module" or .mjs files). CommonJS is unaffected.
  • Static imports (import ... from './foo') have the same limitation, not just dynamic import(); import hoisting also causes another common issue — dotenv runs after the import chain, leaving env vars undefined
  • Using tsx or ts-node in development won't show this error (they auto-resolve extensions), but node dist/server.js in production will fail.
  • PM2 crash loop signature: restart count (↺) keeps growing, uptime never exceeds a few minutes.

Chrome Extension Processing Messages Twice After Hot Reload? WXT HMR Stacks Listeners

· 2 min read

Encountered this issue while building an e-commerce data collection Chrome extension for a client. Here's the root cause and solution.

TL;DR​

WXT framework's HMR re-executes content scripts on file changes but doesn't clean up old window.addEventListener('message', ...) handlers. Each hot reload adds another listener — every postMessage fires all instances.

Fix: Before registering a new listener, retrieve the old one from a window variable and call removeEventListener.

Problem​

Browser console showed each message caught by two different instances:

content.js:114 [CCL] CCL_SHOP_REPORT_DAILY daily caught - instance: nxctn6
content.js:2 [CCL] CCL_SHOP_REPORT_DAILY daily caught - instance: t6jce7

Every postMessage processed twice, causing duplicate requests to the backend.

Root Cause​

WXT (a Vite-based Chrome extension framework) in dev mode triggers HMR on content script changes:

  1. The new content script module loads and executes
  2. A new window.addEventListener('message', messageListener) is registered
  3. The old listener function remains in memory — HMR doesn't clean up DOM event listeners

Result: multiple independent message listeners on window, each postMessage triggering all of them.

Solution​

At the content script entry point, remove the old listener before registering the new one:

const instanceId = Math.random().toString(36).slice(2, 8);

// Retrieve old listener reference
const prevListener = (window as any).__cclMessageListener;
if (prevListener) {
window.removeEventListener('message', prevListener);
}

// Define new listener
const messageListener = (event: MessageEvent) => {
// ... handling logic
};

// Store current reference (for next HMR cycle)
(window as any).__cclMessageListener = messageListener;

// Register
window.addEventListener('message', messageListener);

Key insight: removeEventListener requires the exact same function reference as addEventListener. By storing the function on window, the next HMR cycle can retrieve and remove the old one correctly. This ensures only one active message listener exists at any time, regardless of how many hot reloads occur.

If you're also seeing postMessage data leaking to third-party iframes or empty collected data due to cookie format mismatch, check those issues too.

Note

  • This issue isn't limited to WXT — any framework that hot-reloads content scripts (Plasmo, CRXJS, etc.) can encounter it
  • window variables persist until page refresh; HMR only replaces script modules, not window properties
  • Production builds don't have this issue (content script loads once), but it causes hard-to-debug duplicate requests during development