Akom's Tech Ruminations

Various tech outbursts - code and solutions to practical problems

Docker: Automatically remove containers that have been running too long

Posted by Admin • Thursday, October 20. 2016 • Category: DevOps, Linux

Why Because my Jenkins setup sometimes starts containers and forgets about them. Either it thinks it failed to start one, or the container itself has trouble starting. Either way, I'm left with containers that are running, trying to connect to Jenkins in vain, forever. The proper way to fix this is probably to have the containers timeout at some point, but that mechanism is broken.

Anyway, the fix I have is a true hack: find containers that have been up more than 2 days and kill them. None of our jobs should run for more than about a day, so this is a safe limit. Here is a bash script to do this:

The only hard part here is converting a human-readable age from docker output (32 hours) to a number we can use for comparison. Fortunately, passing "32 hours" to "date -d" gives you a time 32 hours in the future, so from there it's easy.


DOCKER_CMD="docker -H tcp://localhost:YOUR_PORT"
# max seconds before containers should be killed
KILLTHRESHOLD=${1:-172000} #default max age in seconds

if [ "$KILLTHRESHOLD" -gt 0 ] ; then

    $DOCKER_CMD ps --format "{{.ID}}:{{.RunningFor}}" | while read line ; do
        ID=$(echo $line | cut -d : -f 1)
        HUMANAGE="$(echo $line | cut -d : -f 2)"
        FAKETIMESTAMP=$(date -d "$HUMANAGE" +%s)
        AGE=$(echo "$FAKETIMESTAMP - $(date +%s)" | bc)
#        echo "Age is $AGE for $ID"
        if [ "$AGE" -gt "$KILLTHRESHOLD" ] ; then
            logger -t dockerwatch "Killing $ID, running for $HUMANAGE"
            $DOCKER_CMD stop "$ID"
            sleep 5
            $DOCKER_CMD kill "$ID"
            $DOCKER_CMD rm   "$ID"


0 Trackbacks

  1. No Trackbacks


Display comments as (Linear | Threaded)
  1. No comments

Add Comment

You can use [geshi lang=lang_name [,ln={y|n}]][/geshi] tags to embed source code snippets.
Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
Standard emoticons like :-) and ;-) are converted to images.
Markdown format allowed

Submitted comments will be subject to moderation before being displayed.