2 posts tagged with "Deployment"

A Vite project works perfectly in local development. After npm run build and deploying to the server, pages load fine, but every /api request returns 404 — or something sneakier: the response is index.html, and the frontend crashes parsing HTML as JSON.

Encountered this while building an AI Agent SaaS platform for a client with a separated frontend/backend deployment — recording the root cause and the fix.

TL;DR​

server.proxy in vite.config.ts belongs to the dev server only. npm run build produces plain static files with no proxy layer at all; in production, Nginx must take over /api:

location /api {
    proxy_pass http://127.0.0.1:3000;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
}

Placed outside the reach of the SPA fallback (try_files ... /index.html), validated with nginx -t, applied with nginx -s reload.

Symptoms​

Two presentations, one root cause:

Presentation 1: plain 404

GET https://example.com/api/agents  → 404 Not Found

Presentation 2: index.html comes back (sneakier)

GET /api/agents  → 200 OK, but the body is <!DOCTYPE html>...
Frontend fails with: Unexpected token '<' in JSON

The second one is the SPA fallback catching the request: Nginx finds no static file at /api/agents, and try_files $uri /index.html bounces it to the frontend entry page. The status code is even a green 200 — it only blows up when the frontend parses the body, which makes it look like "the request succeeded".

Root Cause: proxy Is a Dev-Server Runtime Feature​

This block in vite.config.ts:

// vite.config.ts
export default defineConfig({
  server: {
    proxy: {
      '/api': 'http://127.0.0.1:3000',
    },
  },
})

works in exactly one context: the dev server started by npm run dev. That dev server is a Node process, and server.proxy is forwarding middleware inside it — the browser requests localhost:5173/api/agents, the dev server passes it to 127.0.0.1:3000, and brings the response back.

After npm run build, the output is a pile of static files in dist/, and the browser fetches them straight from Nginx — there is no Vite process anywhere in that chain, and the server.proxy config stays behind in vite.config.ts; it never ships with the build.

So in production the request path is: browser → Nginx → ?. Nginx matches locations; with no /api rule, no static file exists at /api/agents, so it either 404s or falls through to index.html via the SPA fallback — the two presentations, explained.

The Fix: Let Nginx Own the Production Proxy​

Step 1: add a proxying location to the server block.

server {
    listen 80;
    server_name example.com;
    root /var/www/app/dist;

    # API proxy: prefix match, wins over the SPA fallback
    location /api {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # SPA fallback: everything else goes to the frontend entry
    location / {
        try_files $uri $uri/ /index.html;
    }
}

Two points:

• proxy_pass points at the real backend address and port (swap the example's 3000 for yours); /api-prefixed requests are forwarded as-is, so backend routes must carry the /api prefix
• Nginx prefix locations match by longest prefix, so /api naturally beats / — no ordering tricks needed; just make sure your longest API common prefix is covered

Step 2: verify, then serve traffic.

nginx -t          # syntax check
nginx -s reload   # graceful reload

# Smoke test: should return backend data, not HTML
curl -i https://example.com/api/health

JSON with Content-Type: application/json means the proxy is live; text/html means the fallback is still catching requests.

A side benefit: once both apps share a domain through the reverse proxy, the CORS problem you dodged in dev with Vite proxy disappears in production too — no CORS headers to configure.

This post and Frontend Deploy Looks Outdated? Nginx Cache and Build Output Checks are companion reads: that one covers "the deployed code is stale", this one covers "the API has no proxy layer in production". Check both and deployment-related 404s are pretty much exhausted.

FAQ​

Does Vite proxy work in production?​

No. server.proxy is a runtime middleware of the dev server — it exists when you run npm run dev. A production build produces plain static files served by Nginx (or any web server), with no Vite process and no proxy layer. The production equivalent is an Nginx location block with proxy_pass, which typically adds 2 proxy_set_header lines and a reload.

Why is the Vite proxy not working after build?​

Because the proxy config never ships with the build. After deploying, /api requests hit Nginx directly; with no /api location rule they either 404 or get caught by the SPA fallback (try_files ... /index.html), returning HTML with a 200 status that breaks JSON parsing on the frontend. Add an /api proxy location and reload Nginx.

How do I route API requests through Nginx for a Vite app?​

Inside the server block add: location /api { proxy_pass http://127.0.0.1:3000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } — then nginx -t and nginx -s reload. Verify with curl -i https://example.com/api/health: a JSON content type means the proxy is live, text/html means the SPA fallback is still catching it.

Push code, the GitHub Actions deploy step turns red and exits — yet the service on the server is already the latest version. Another day, the reverse: everything is green locally while pnpm install --frozen-lockfile fails on CI every single time. These two opposite signal failures both live in the deploy pipeline, not in the code.

I hit this while building an e-commerce automated data collection tool for a client — bulk-scraping product images, SKUs, prices, and reviews, cleaned and exported as structured data for inventory management and competitor analysis. The tool's server deploys through PM2 + GitHub Actions, and both false failures happened on that pipeline.

TL;DR​

• CI red but the deploy actually landed: two deploy channels ran pm2 restart concurrently; one looked up the process mid-restart and misreported failure. Signature: pm2.log shows a successful restart and a process already online error within the same second. Fix: collapse deploys to a single channel.
• Local green but CI always fails: pnpm add ran in a subdirectory, updating only the child package.json while the root pnpm-lock.yaml went stale. Fix: run pnpm install at the workspace root and commit the regenerated lockfile.

Scenario 1: green locally, CI always red — pnpm lockfile drift​

This failure has nothing to do with code quality — pnpm-lock.yaml simply drifted out of sync with package.json, and CI is the only environment that checks the two strictly.

The symptom​

CI fails on every run at the same step:

ERR_PNPM_OUTDATED_LOCKFILE

Locally, install, build, and tests all pass. That "works on my machine but CI fails" combination tempts you to suspect CI caching or the Node version first — both wrong here.

Root cause: pnpm add ran in the wrong directory​

This project is a pnpm workspace monorepo with a single lockfile at the root. The dependency was added like this:

# executed inside client/
pnpm --filter @ccl-ext/client add <pkg>

client/package.json got updated, but the root pnpm-lock.yaml was never regenerated and committed. CI then received a lockfile inconsistent with package.json, and the --frozen-lockfile check refused to install.

Why local never catches it: node_modules already has the packages physically installed, so a local install reuses what is there and never exercises the frozen check. CI starts from a clean environment and compares the lockfile strictly, every time.

There is also a telltale side effect: running pnpm in the wrong cwd leaves a stray client/pnpm-lock.yaml behind. Spotting that file is near-proof that a command ran in the wrong place again.

The fix​

Two steps:

1. From the workspace root, run pnpm install to regenerate the root lockfile, and commit it with the code;
2. From now on, run pnpm add / pnpm remove at the workspace root only.

The specifiers diff in the CI failure details names exactly which package's dependency range changed — use it to confirm the fix targets the right spot.

A similar "wrong attribution" issue in multi-package workspaces is covered here: npm audit blames the wrong directory? Multi-package deploys audit N package trees.

Scenario 2: Actions reports failure, the deploy actually landed — pm2 restart race​

The deploy itself did not fail; the second, colliding restart did — PM2 reported that race as a deploy error.

The symptom​

After a push, GitHub Actions "Deploy Server" fails at the pm2 restart step:

[PM2][ERROR] Process 3 not found → exit 1

But on the server: the code is the latest, pm2 list shows the process online, and the health check returns 200. All three deploy essentials pass — only Actions believes it failed.

Root cause: two channels restarting the same process​

Deploys had two trigger paths at the time:

1. The push trigger in deploy.yml;
2. A local /deploy script (deploy.sh).

One push made both paths run pm2 restart ccl-ext-api. When the restarts collided, one of them queried the process at the exact instant the other was replacing it, found nothing, logged Process not found, and exited 1 — PM2 reported the race as a failure.

Verification: the same second in pm2.log​

pm2.log is the hardest evidence. The log shows two kinds of records within the same second:

# one side: the restart completes, process online
Stopping → starting → online

# the other: the colliding restart finds no process
PM2 error: process already online

One restart "in flight" interleaving with another "querying" inside 1 second is the race signature. Combined with the three checks — code latest, process online, health 200 — the deploy landed and the Actions failure was noise.

The fix: drop the push trigger, single deploy channel​

The change removes the push auto-trigger from deploy.yml, keeping workflow_dispatch for manual fallback:

on:
  workflow_dispatch:

Between the two options — a concurrency lock versus collapsing to one channel — I chose the latter: a lock only makes the two channels queue up, leaving two deploy paths in place. A deploy should have exactly one entry point; who deployed what, when, should originate from a single place. After the change, the false alarms never returned.

For a different take on verifying what is actually live after a deploy, see: Frontend deployed but the site did not update? Troubleshooting stale builds.

Side by side: both false failures share one trait — a polluted signal source​

Put the two scenarios together and one trait surfaces immediately: what broke was never the deploy result, but the pipeline producing the signal.

CI's red and green are just the pipeline's output. When the pipeline itself is polluted (two lockfiles, two channels), the signal stops being trustworthy. Fix the pipeline first; read the signal after.

FAQ​

Why does pm2 report 'Process N not found' during a GitHub Actions deploy?​

Two deploy channels ran pm2 restart on the same process at the same time. One restart hits the instant the process is being replaced, finds nothing, and exits 1 — while pm2.log shows a successful restart and a 'process already online' error within the same second.

GitHub Actions shows the deploy as failed — how do I tell if it is a false alarm?​

Check 3 things: the code on the server is the latest, pm2 list shows the process online, and the health check returns 200. If all three pass and the failure point is the pm2 restart step, it is a concurrency false alarm; collapsing deploys to a single channel removes it.

What causes a pnpm frozen-lockfile error in CI when local installs pass?​

The root cause is pnpm-lock.yaml out of sync with package.json: running pnpm add in a subdirectory updates the child package declaration but leaves the root lockfile stale. Run pnpm install once at the workspace root, commit the regenerated lockfile, and check the specifiers diff in the CI log.