Docker compose container knowledge 101

 what requirements.txt Does

requirements.txt is a plain text file that lists all the Python packages your project depends on.

When you run:

bash

Copy

Edit

pip install -r requirements.txt

---------------------------------------------------------------------------------

docker-compose up -d starts the crawler container in the background (idling with tail -f /dev/null).

When you run:

docker-compose run --rm crawler python crawler.py

it spins up a second, temporary container with the same image and runs the script — and then removes it afterward (--rm).

So yes, you'll briefly have two containers:

One idle (crawler)

One executing your script (docker-compose run instance)

---------------------------------------------------------------

command: tail -f /dev/null  # Keeps container idle in docker-compose.yml file what does it do ?

is a common Docker trick that means:

"Keep the container running, but do nothing useful."

---

🧠 Why would you do this?

Because by default, Docker containers exit immediately after their main process finishes.
So if your service doesn't do anything (e.g., no long-running server or script), it will just shut down.

By running:

tail -f /dev/null

You:

• Start a "fake" long-running process (like watching an empty file forever)

• Prevent the container from exiting

• Allow yourself to exec into the container or manually run things later