semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown len(cache))

See original GitHub issue

Bug Description

Hi, everyone! When I try to use autokeras fit some data and label, I encounter such warnings as: “semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown len(cache))

After some iterations, for example, after prompt “Model 9”, the program is hanging due to insufficient memory.

I don’t think it has connection with my code, here is my code:

from autokeras.image_supervised import ImageClassifier
import numpy as np

if __name__ == '__main__':
    x_train = np.load('x_train.npy')
    y_train = np.load('y_train.npy')
    x_test = np.load('x_test.npy')
    y_test = np.load('y_test.npy')
    x_train = x_train.reshape((-1, 32, 32, 3))
    y_train.reshape((-1, 1))
    x_test = x_test.reshape((-1, 32, 32, 3))
    y_test = y_test.reshape((-1, 1))
    clf = ImageClassifier(verbose=True)
    clf.fit(x_train, y_train, time_limit=12*60*60)
    clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
    pred = clf.evaluate(x_test, y_test)
    print(pred)

The magnitude of my dataset is similar to CIFAR10.

I want to know, how to fix this warning and following memory issue? Does anyone has experience in fixing such things? Thanks in advance.

Reproducing Steps

Steps to reproduce the behavior:

  • Step 1: …
  • Step 2: …

Expected Behavior

Setup Details

Include the details about the versions of:

  • OS type and version: Ubuntu 16.04
  • Python: 3.6
  • autokeras:
  • scikit-learn:
  • numpy:
  • keras: 2.2.2
  • scipy:
  • tensorflow: 1.10.0
  • pytorch: All requirements are installed by “pip install autokeras” and I run my code in a Docker instance, Some info on my Docker environment: chuanming@Galadriel:~$ nvidia-docker version NVIDIA Docker: 2.0.3 Client: Version: 18.03.1-ce API version: 1.37 Go version: go1.9.5 Git commit: 9ee9f40 Built: Thu Apr 26 07:17:20 2018 OS/Arch: linux/amd64 Experimental: false Orchestrator: swarm

Server: Engine: Version: 18.03.1-ce API version: 1.37 (minimum version 1.12) Go version: go1.9.5 Git commit: 9ee9f40 Built: Thu Apr 26 07:15:30 2018 OS/Arch: linux/amd64 Experimental: false

Additional context

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:5
  • Comments:5

github_iconTop GitHub Comments

4reactions
monikatomar92commented, Sep 25, 2018

I experienced the same warning. UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown len(cache))

1reaction
anirban0104commented, Jul 19, 2021

I am getting this same error while running a ML model in deepfacelab colab, and after the error, the cell stops executing.

Read more comments on GitHub >

github_iconTop Results From Across the Web

UserWarning: semaphore_tracker: There appear to be 34 ...
This started showing up all of a sudden in the last two days, ... appear to be 34 leaked semaphores to clean up...
Read more >
Issue with multiprocessing semaphore tracking
... UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown len(cache)). I didn't face this issue a ...
Read more >
There appear to be 6 leaked semaphore objects to clean up at ...
I'm trying a test connection on my Firebase Realtime database via python 3.8. I have two scripts, one ...
Read more >
[Python] Processes killed and semaphore objects leaked ...
[Python] Processes killed and semaphore objects leaked when ... There appear to be 1 leaked semaphore objects to clean up at shutdown ......
Read more >
There appear to be 1 leaked semaphores to clean ... - iTecNote
Semaphore_tracker : There appear to be 1 leaked semaphores to clean up at shutdown len(cache)) ... I was run an autokeras code via...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found