Stay Hungry

Monday, 27 July 2026

GO OS process vs thread

## 1. Channels vs Goroutines vs Processes

These are 3 different things:

```javascript

┌─── OS Process (itemgen-service binary) ─────────────────────┐

│ ┌─ OS Thread 1 ────────────────────────────────────────┐ │

│ │ ┌─ Goroutine: HTTP handler ──┐ │ │

│ │ │ ┌─ Goroutine: HTTP handler ┤ │ │

│ │ │ │ ┌─ Goroutine: Worker ───┤ │ │

│ │ │ │ │ │ │ │

│ │ │ │ │ ←── channel (in-memory) ──┘ │ │

│ │ │ │ │ (just a queue, NOT a goroutine) │ │

│ │ └──┴──┴────────────────────────┘ │ │

│ └───────────────────────────────────────────────────────┘ │

│ ┌─ OS Thread 2 ────────────────────────────────────────┐ │

│ │ ┌─ Goroutine: scheduler ──┐ │ │

│ │ └──────────────────────────┘ │ │

│ └───────────────────────────────────────────────────────┘ │

└──────────────────────────────────────────────────────────────┘

```

- __Channel__: Just a thread-safe in-memory queue. No code runs "inside" it. It's a data structure, like a slice with a lock.

- __Goroutine__: A lightweight unit of execution (~2KB stack). NOT an OS process. Go runs thousands of goroutines on a few OS threads. Your app already uses goroutines (the scheduler, the item generation `sync.WaitGroup` workers, etc.)

- __Process__: The OS-level process — your running `itemgen-service` binary. There's only ONE process. Everything (channels, goroutines) lives inside it.

```javascript

$ ps aux | grep itemgen-service

jxiang 12345 ... itemgen-service ← ONE process, always

$ htop (press H to toggle thread view)

12345 itemgen-service ← main thread

12346 itemgen-service ← OS thread (Go runtime)

12347 itemgen-service ← OS thread (Go runtime)

12348 itemgen-service ← OS thread (Go runtime)

```

They all share PID 12345 as the parent. The sub-IDs (12346, 12347...) are __threads__, not separate processes. They share the same memory space.

## Go's GMP model

```javascript

Goroutines (G) OS Threads (M) CPU Cores (P)

┌──────────────┐ ┌──────────────┐ ┌──────────┐

│ HTTP handler │──┐ │ │ │ │

│ HTTP handler │──┤ │ Thread 1 │◀────▶│ Core 1 │

│ Scheduler │──┼──▶ │ │ │ │

│ DB sync │──┤ ├──────────────┤ ├──────────┤

│ Worker │──┤ │ Thread 2 │◀────▶│ Core 2 │

│ item gen #1 │──┘ │ │ │ │

│ item gen #2 │ └──────────────┘ └──────────┘

└──────────────┘

~1000 goroutines ~4 threads (= CPU count)

```

- __Goroutines__: Your code spawns these with `go func()`. Thousands can exist. They're cheap (~2KB each).

- __OS Threads__: Go runtime creates these automatically. Default = number of CPU cores. These are what `htop` shows.

- __Processes__: Only ONE. Your binary. Always.

## How to verify

```bash

# Count OS processes (always 1):

ps -C itemgen-service -o pid= | wc -l

# Output: 1

# Count OS threads (several):

ps -C ite,gen-service -L -o tid= | wc -l

# Output: 4-8 (depending on CPU cores)

# Count goroutines (many):

# Run inside your Go code:

runtime.NumGoroutine()

# Output: 50-100+ (handlers, scheduler, worker, etc.)

```

## Why you might see "new process spawning"

Some monitoring tools (like Docker stats, certain dashboards) display __threads as if they were processes__. On Linux, `/proc/<pid>/task/` lists each thread with its own TID (thread ID), which can look like process IDs. But they're threads — they share memory, file descriptors, and the same PID.

Your app already spawns goroutines today (scheduler, item generation workers, DB sync) — you're just not noticing because the worker pattern is the same. Adding the notify worker is no different.

Shall I implement the SMTP notification plan now?

# Linux — count threads vs processes:

ps -eLf | grep itemgen-service | head

# Columns: UID PID PPID LWP C NLWP ...

# PID = process ID (same for all)

# LWP = thread ID (different for each)

# NLWP = number of threads

# Or simpler:

cat /proc/$(pgrep itemgen-service)/status | grep Threads

# Output: Threads: 7

# On Windows (your machine):

tasklist /fi "imagename eq itemgen-service.exe" /fo csv

# Shows 1 process

# PowerShell — count threads:

(Get-Process itemgen-service).Threads.Count

# Output: 7 (threads, not processes)

Thursday, 16 July 2026

timezone handling guide

# Timezone Handling Guide

## 1. Best Practice: Store Time in UTC

**Keep all timestamps in UTC in the database.** This is the industry standard for servers.

| Component | Current Setting | Why It's Best |

|-----------|----------------|---------------|

| MySQL (Docker) | UTC (default) | No DST issues, universal, sync works across timezones |

| Go application | UTC (no `loc=` in DSN) | Consistent with MySQL, no conversion bugs |

| Source DB sync | UTC | `TIMESTAMP` stored as UTC internally, comparisons are correct |

**MySQL `TIMESTAMP` is always stored as UTC internally.** The display timezone only affects what you see, not what's stored.

---

## 2. Direct DB Query: Use `SET time_zone` Per Connection

When querying the database manually (MySQL CLI, Workbench, etc.):

```sql

-- Set timezone for this session

SET time_zone = 'America/Vancouver';

-- All TIMESTAMP columns now display in PST

SELECT * FROM sync_metadata;

-- Shows: 2026-07-16 11:31:37 (PST display)

-- Stored: 2026-07-16 18:31:37 (UTC — unchanged)

-- Query with PST times — MySQL converts automatically

SELECT * FROM sync_metadata WHERE last_run_at > '2026-07-16 11:00:00';

```

**Key points:**

- ✅ Display only — does NOT change stored UTC data

- ✅ Lasts for the entire session (all queries until you disconnect)

- ✅ Resets to UTC when connection closes

- ✅ Does NOT affect the Go application (separate connections)

- ⚠️ Only works for `TIMESTAMP` columns, not `DATETIME`

---

## 3. Why `SET time_zone` Can't Be Used in Go

Go uses a **connection pool** — connections are reused, not created/destroyed per query. If you `SET time_zone` on a connection, it **leaks** to the next function that gets the same connection.

### Connection Pool Diagram

```

┌─────────────────────────────────────────────────────────────────┐

│ Connection Pool (max 100 open, 50 idle, 300s lifetime) │

│ │

│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │

│ │ conn1 │ │ conn2 │ │ conn3 │ │ ... │ ← 50 idle │

│ │ (UTC) │ │ (UTC) │ │ (UTC) │ │ (UTC) │ (pre-warmed) │

│ └────────┘ └────────┘ └────────┘ └────────┘ │

└─────────────────────────────────────────────────────────────────┘

Function A: SET time_zone on conn1

│ │

│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │

│ │ conn1 │ │ conn2 │ │ conn3 │ │ ... │ │

│ │ (PST!) │ │ (UTC) │ │ (UTC) │ │ (UTC) │ │

│ └───┬────┘ └────────┘ └────────┘ └────────┘ │

│ │ │

│ └── Function A done, conn1 returned to pool │

│ conn1 STILL has PST! │

Function B: gets conn1 from pool

│ │

│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │

│ │ conn1 │ │ conn2 │ │ conn3 │ │ ... │ │

│ │ (PST!) │ │ (UTC) │ │ (UTC) │ │ (UTC) │ │

│ └───┬────┘ └────────┘ └────────┘ └────────┘ │

│ │ │

│ └── Function B gets conn1 — INHERITS PST! ← LEAKED! │

│ B's query runs in PST, not UTC │

conn1 stays PST until:

- Connection recycled (after 300s lifetime)

- Explicitly reset with SET time_zone = 'UTC'

- Application restarts

```

### Pool Configuration (from `.env`)

```

DB_MAX_OPEN_CONNS=100 Max simultaneous connections

DB_MAX_IDLE_CONNS=50 Idle connections kept alive (pre-warmed)

DB_CONN_MAX_LIFETIME_SEC=300 Each connection recycled after 5 minutes

```

At rest: 50 idle connections waiting. Under load: up to 100 total. When load drops: extras closed back to 50.

### Why It Leaks

| What you think | What actually happens |

|---|---|

| Function ends → connection closed → timezone gone | Function ends → connection returned to pool → timezone persists |

| Next function gets fresh UTC connection | Next function might get the same PST connection |

| Each function is isolated | Timezone leaks for up to 300 seconds (connection lifetime) |

---

## 4. Solution: Convert Time in Go Code

### The Pattern

```

Input (PST) → Convert to UTC → Query DB (UTC) → Results (UTC) → Convert to PST → Display

```

### In Go Code

```go

// Load Vancouver timezone (handles PST/PDT automatically)

pstLoc, _ := time.LoadLocation("America/Vancouver")

// 1. Input: user wants logs after "2026-07-16 11:00:00" PST

pstTime, _ := time.ParseInLocation("2006-01-02 15:04:05", "2026-07-16 11:00:00", pstLoc)

// 2. Convert to UTC for query

utcTime := pstTime.UTC() // 2026-07-16 18:00:00 UTC

// 3. Query DB with UTC — no SET time_zone needed, no pool contamination

rows, err := db.Query(`

SELECT last_run_at, last_sync_time, last_run_status

FROM sync_metadata

WHERE last_run_at > ?

`, utcTime)

// 4. Read results (Go reads TIMESTAMP as UTC because no loc= in DSN)

for rows.Next() {

var lastRunAt time.Time // Go gives you UTC time

var lastSyncTime time.Time

var status string

rows.Scan(&lastRunAt, &lastSyncTime, &status)

// 5. Convert to PST for display

pstRunAt := lastRunAt.In(pstLoc) // 2026-07-16 11:31:37 PST

fmt.Printf("Run at: %s, Status: %s\n",

pstRunAt.Format("2006-01-02 15:04:05"), status)

}

```

### Why This Is Cost-Free

`time.Time.In(loc)` doesn't do any calculation — it just attaches the timezone location to the existing time value. The actual time moment doesn't change, only how it's displayed.

| Operation | Time per row | Comparison |

|---|---|---|

| `rows.Scan()` (parse from MySQL) | ~1-2 microseconds | 1,000x slower than conversion |

| Network I/O (fetching the row) | ~50-100 microseconds | 50,000x slower than conversion |

| `time.In(pstLoc)` (timezone conversion) | ~1-2 nanoseconds | Baseline — essentially free |

Even with 10,000 rows:

- Go conversion: 10,000 × 1ns = **0.01 milliseconds**

- MySQL fetching: 10,000 × 100μs = **1 second**

The timezone conversion is **0.001%** of the total time. Negligible.

### Why This Is Better Than Alternatives

|---|---|---|---|---|

---

## 5. If You Ever Want to Change to PST (All or Nothing)

If you decide to move away from UTC, **all three changes must be made together**:

### Change 1: docker-compose.yml — both containers

```yaml

certgen:

environment:

- TZ=America/Vancouver # Go's time.Now() returns PST

mysql:

environment:

- TZ=America/Vancouver # MySQL's CURRENT_TIMESTAMP returns PST

```

### Change 2: db.go — local DB DSN (add `loc=Local`)

```go

dsn := "%s:%s@tcp(%s:%d)/%s?parseTime=true&loc=Local&multiStatements=true&timeout=10s"

```

### Change 3: db.go — source DB DSN (add `loc=Local`)

```go

dsn := "%s:%s@tcp(%s:%d)/%s?parseTime=true&loc=Local&timeout=10s"

```

### Why All Three

| If you change... | But not... | What breaks |

|---|---|---|

| `TZ` on both containers | DSN `loc=Local` | Go reads TIMESTAMP as UTC — mismatch with MySQL PST |

| DSN `loc=Local` | `TZ` on containers | Go uses container's UTC — `loc=Local` = UTC, no effect |

| MySQL `TZ` only | Go `TZ` + DSN | `CURRENT_TIMESTAMP` = PST, `time.Now()` = UTC — mismatch |

| Go `TZ` only | MySQL `TZ` + DSN | `time.Now()` = PST, `CURRENT_TIMESTAMP` = UTC — mismatch |

**All three or none. Partial changes cause mismatches.**

### Risks of Changing to PST

- ❌ Daylight saving time: March/November clock changes cause gaps or duplicates in sync

- ❌ Source DB must also be PST, or sync breaks

- ❌ If source DB is UTC, `modify_time` comparison is off by 7-8 hours

---

## 6. Quick Reference

| Scenario | What to do |

|---|---|

| **Keep current setup (recommended)** | Change nothing. Everything is UTC. |

| **View timestamps in PST manually** | `SET time_zone = 'America/Vancouver';` per MySQL session |

| **Query with PST time in Go** | Convert PST→UTC before query, convert UTC→PST after |

| **Change everything to PST** | Set `TZ` on both containers + `loc=Local` in both DSNs |

| **Check if a column auto-converts** | `TIMESTAMP` = yes, `DATETIME` = no |

Friday, 10 July 2026

GO LANG: GIN panic recovery

Case 1: Panic in an HTTP handler (e.g., a request triggers a panic)

RecoveryMiddleware catches it:

func RecoveryMiddleware() gin.HandlerFunc {

return func(c *gin.Context) {

defer func() {

if rec := recover(); rec != nil {

logger.Error("panic recovered", "panic", rec, ...)

c.AbortWithStatusJSON(http.StatusInternalServerError, ...)

}

}()

c.Next()

}

✅ The process keeps running — only that one request fails with 500

✅ Other requests continue normally

❌ Container does not restart (no need to)

This is handled correctly in your code

Case 2: Panic in a goroutine (e.g., scheduler, sync, )

Goroutine panics are not caught by RecoveryMiddleware (it only covers HTTP handlers). If a goroutine panics:

The entire process crashes (Go runtime kills the process on unrecovered goroutine panic)

Container exits with non-zero code

With restart: unless-stopped:

✅ Docker detects the container exited

✅ Docker restarts it automatically

✅ Service comes back up in ~5-10 seconds

Case 3: Process killed by OOM (out of memory)

If the VM runs out of memory, the OS kills the process:

Container exits

restart: unless-stopped restarts it

Case 4: Infinite loop / CPU spike (not a crash)

If a bug causes 100% CPU but no crash:

Container stays running

Docker does not restart it (it's still "running")

You'd need a health check to detect this

Your current protection:

Scenario Caught by middleware? Container crashes? Auto-restarts?

Panic in HTTP handler ✅ Yes ❌ No N/A

Panic in goroutine ❌ No ✅ Yes ✅ Yes (with restart policy)

OOM kill N/A ✅ Yes ✅ Yes (with restart policy)

Process segfault N/A ✅ Yes ✅ Yes (with restart policy)

Bottom line: Once you add restart: unless-stopped to <yourService>, the container will auto-restart on any process crash. The RecoveryMiddleware prevents most crashes by catching handler panics. The restart policy is the safety net for everything else.

docker compose restart:always restart:unless stopped notes, robuts design

The only difference is what happens when you manually stop a container:

Scenario restart: always restart: unless-stopped

Container crashes ✅ Restarts ✅ Restarts

VM reboots ✅ Restarts ✅ Restarts

You run docker stop ✅ Restarts anyway ❌ Stays stopped

You run docker compose down ❌ Stays down (compose down overrides) ❌ Stays down

The key difference:

always — if you manually docker stop certgen, it will restart itself immediately. You can't keep it stopped unless you remove the container or use docker compose down.

unless-stopped — if you manually docker stop certgen, it stays stopped. It remembers "the user stopped me" and won't restart until you explicitly start it again.

************************************************************************

1. __`depends_on` controls order__ — yes, the dependency starts first. In your case, MySQL starts before <your_service>. ✅

2. __No `depends_on`__ — Docker Compose starts services in __the order they appear in the file__ (top to bottom). So if you remove `depends_on`, <your_service>(listed first) would start before MySQL (listed second).

__But there's an important caveat:__

`depends_on` only waits for the container to __start__, not for the service inside to be __ready__. So even with `depends_on`, MySQL container starts before certgen, but MySQL might not be ready to accept connections yet (it takes 3-10 seconds to initialize).

That's why your `db.Connect()` retry logic (30 retries × 2s) is essential — it bridges the gap between "container started" and "MySQL ready to accept connections."

__Summary:__

Thursday, 12 March 2026

Go LANG unit test

https://www.freecodecamp.org/news/unit-testing-in-go-a-beginners-guide/

In Memory unit test:

$ go test -v
...
=== RUN   TestSafeDivide
=== RUN   TestSafeDivide/normal_division
=== RUN   TestSafeDivide/division_by_zero
--- PASS: TestSafeDivide (0.00s)
    --- PASS: TestSafeDivide/normal_division (0.00s)
    --- PASS: TestSafeDivide/division_by_zero (0.00s)
PASS
ok      _/C_/projects/Articles/Go_Testing       0.323s

RXJS REDUX

https://github.com/ngrx/platform/issues/162

Order guaranteed,

same action,

reducer always execute first then effects

Bash $?, and exit code

Bash $?, exist code

every command executed by the shell ends with an exit code, not just Bash scripts.

Even though npm run lint is not a Bash script, it is still a process executed by the shell, and every process on Unix-like systems (Linux, macOS, etc.) returns an exit status when it finishes.

How the chain works

When you run:


npm run lint

this happens under the hood:

Bash launches the program npm.
npm reads package.json and finds the lint script.
npm launches the command defined there (for example eslint .).
That program finishes and returns an exit code.
npm forwards that exit code back to the shell.
Bash stores it in $?.

Example package.json:

In Bash, the key piece is this line:


STATUS=$?

$? is a special Bash variable that always contains the exit code of the last command that ran.

Step-by-step what happens

Change directory:


cd cli

Run the lint script:


npm run lint

This runs the lint script defined in your package.json.
When it finishes, it exits with a status code:
- 0 → success
- non-zero (e.g., 1) → failure

Capture that exit code:


STATUS=$?

$? now holds the exit code returned by npm run lint.
That value gets stored in the variable STATUS.

Check if it failed:


if [ $STATUS -ne 0 ]; then

-ne means not equal.
So if the exit code is not 0, the script treats it as a failure.

Abort the commit:


echo "Lint failed. Commit aborted."
exit 1

Otherwise:


echo "Lint passed."
exit 0

Example

If your lint script is something like:


"scripts": {
  "lint": "eslint ."
}

If eslint finds errors → exits 1 → $? becomes 1 → commit aborted.
If no errors → exits 0 → $? becomes 0 → commit continues.

Simpler equivalent

You can write the same logic more directly:


cd cli

if ! npm run lint; then
  echo "Lint failed. Commit aborted."
  exit 1
fi

echo "Lint passed."
exit 0

Here if ! command automatically checks the exit code.