Docker container with status "Dead" after consul healthcheck runs -


i using consul's healthcheck feature, , keep getting these these "dead" containers: 

container id  image                   command              created         status              ports                                                                                                                                                                    names 20fd397ba638  progrium/consul:latest  "\"/bin/bash -c 'cur 15 minutes ago  dead

what "dead" container? when stopped container become "dead"?

for record, run progrium/consul + gliderlabs/registrator images + service_xxxx_check env variables health checking. runs healthcheck script running image every x secs, docker run --rm my/img healthcheck.sh

i'm interested in general "dead" means , how prevent happening. peculiar thing dead containers have no name.

this info container inspection:

  "state": {         "dead": true,         "error": "",         "exitcode": 1,         "finishedat": "2015-05-30t19:00:01.814291614z",         "oomkilled": false,         "paused": false,         "pid": 0,         "restarting": false,         "running": false,         "startedat": "2015-05-30t18:59:51.739464262z"     },

the strange thing every , container becomes dead , isn't removed.

thank you

edit: looking @ logs, found makes container stop fail:

  handler delete /containers/{name:.*} returned error: cannot destroy container 003876e41429013e46187ebcf6acce1486bc5011435c610bd163b159ba550fbc:  driver aufs failed remove root filesystem 003876e41429013e46187ebcf6acce1486bc5011435c610bd163b159ba550fbc:  rename /var/lib/docker/aufs/diff/003876e41429013e46187ebcf6acce1486bc5011435c610bd163b159ba550fbc  /var/lib/docker/aufs/ diff/003876e41429013e46187ebcf6acce1486bc5011435c610bd163b159ba550fbc-removing:  device or resource busy

why happen?

edit2: found this: https://github.com/docker/docker/issues/9665

update march 2016: issue 9665 has been closed pr 21107 (for docker 1.11 possibly)
should avoid "driver aufs failed remove root filesystem", "device or resource busy" problem.

original answer may 2015

dead 1 if container states, tested container.start()

if container.removalinprogress || container.dead {         return fmt.errorf("container marked removal , cannot started.") }

it set dead when stopping fails, in order prevent container restarting.

amongst possible cause of failure, see container.kill().
means kill -15 , kill -9 both failing.

// 1. send sigterm if err := container.killpossiblydeadprocess(15); err != nil {     logrus.infof("failed send sigterm process, force killing")     if err := container.killpossiblydeadprocess(9); err != nil {

that mean, op mention, busy device or resource, preventing process killed.


Comments

Post a Comment

Popular posts from this blog

node.js - Using Node without global install -

i'm working on few node & electron (atom shell) projects , i'm curious how them work without user having install node globally. i've seen few applications including node directory in project contains node.exe binary. how go setting node project use binary? edit: thought should add, i've searched everywhere answer this, unrelated results.
Read more

How to access a php class file from PHPFox framework into javascript code written in simple HTML file? -

i've website developed using phpfox v3.0.7 . now i'm trying implement 'sse(server sent events)' 1 of php files. reference i'm giving below necessary code part of file. file titled process.class.php , present @ location /var/www/module/notification/include/service/process.class.php present on server having ip http://34.124.40.142/ . <?php /** * [phpfox_header] */ header('content-type: text/event-stream'); header('cache-control: no-cache'); defined('phpfox') or exit('no dice!'); /** * * * @copyright [phpfox_copyright] * @author raymond benc * @package phpfox_service * @version $id: service.class.php 67 2009-01-20 11:32:45z raymond_benc $ */ class notification_service_process extends phpfox_service { /** * class constructor */ public function __construct() { $this->_stable = phpfox::gett('notification'); } public function add($...
Read more

java - Null response to php query in android, even though php works properly -

Image
table i'm trying retrieve above data using 2 php files below. want pass email parameter , return bets user = email. work fine when tested advanced rest client, problem not php. however, when try through android 2 java classes below, null pointer exception (logcat below) , json object returned null. i'm not sure error is, think i'm not passing parameter php , that's why it's producing null response. logcat 05-31 15:10:50.227 26074-26074/com.example.albert.betterapp e/androidruntime﹕ fatal exception: main process: com.example.albert.betterapp, pid: 26074 java.lang.runtimeexception: unable start activity componentinfo{com.example.albert.betterapp/com.example.albert.betterapp.displayallbets}: java.lang.nullpointerexception @ android.app.activitythread.performlaunchactivity(activitythread.java:2292) @ android.app.activitythread.handlelaunchactivity(activitythread.java:2350) @ android.app.activitythread.acce...
Read more