Model load failed : [StatusCode.INTERNAL]

See original GitHub issue

Hello, I am trying to run the model_analyzer on an xgboost model I have. I am able to load the docker sdk client container and submit a model analyzer command, but I keep getting the following error for all the auto-generated models.

[Model Analyzer] WARNING: Overriding the output model repo path "/home/dvanstee/data/projects/2022-07-tritonDemo/fil_demo/model_repository/ma_out1"
[Model Analyzer] Starting a Triton Server using docker
[Model Analyzer] Loaded checkpoint from file /tmp/ckpts/1.ckpt
[Model Analyzer] Profiling server only metrics...
[Model Analyzer] Stopped Triton Server.
[Model Analyzer] 
[Model Analyzer] Creating model config: xgboost_classifier_config_default
[Model Analyzer] 
[Model Analyzer] Model xgboost_classifier_config_default load failed: [StatusCode.INTERNAL] failed to load 'xgboost_classifier_config_default', failed to poll from model repository

Here is how I start docker

docker run -it --rm --net=host \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v ${BASE}/model_repository:/models \
  -v ${BASE}/model_repository:/${BASE}/model_repository \
  -v ${BASE}:/notebooks/fil_demo/ \
  nvcr.io/nvidia/tritonserver:22.06-py3-sdk

Here is how I run the model analyzer…

model-analyzer profile \
    --checkpoint-directory /tmp/ckpts \
    --model-repository /model_repository \
    --triton-launch-mode=docker \
    --triton-docker-image nvcr.io/nvidia/tritonserver:22.06-py3 \
    --triton-docker-mounts /model_repository:/model_repository:rw \
    --profile-models xgboost_classifier \
    --override-output-model-repository \
    --output-model-repository-path /model_repository/ma_out1

Here is what I see in the output directory

/model_repository/ma_out1$ ll
total 36
drwxrwxrwx 6 dvanstee dvanstee 4096 Aug  1 12:35 ..
drwxr-xr-x 3 root     root     4096 Aug  1 12:35 xgboost_classifier_config_default
drwxr-xr-x 2 root     root     4096 Aug  1 12:35 xgboost_classifier_config_0
drwxr-xr-x 2 root     root     4096 Aug  1 12:35 xgboost_classifier_config_1
drwxr-xr-x 2 root     root     4096 Aug  1 12:35 xgboost_classifier_config_2
drwxr-xr-x 2 root     root     4096 Aug  1 12:35 xgboost_classifier_config_3
drwxr-xr-x 2 root     root     4096 Aug  1 12:35 xgboost_classifier_config_4
drwxr-xr-x 9 root     root     4096 Aug  1 12:35 .
drwxr-xr-x 2 root     root     4096 Aug  1 12:35 xgboost_classifier_config_5

I am able to run inference, and also use perf_analyzer, but I cant seem to figure out the right syntax for model_analyzer.
I was wondering if anyone can see why I cant seem to get the models that are automatically generated to load. Any ideas for me ?

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:2
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
tgerdesnvcommented, Sep 12, 2022

Yes, this is the exact same issue. You can’t do this: -v /home/dieter/Documents/triton_analyzer/examples/quick-start:/models \

You need to do this: -v /home/dieter/Documents/triton_analyzer/examples/quick-start:/home/dieter/Documents/triton_analyzer/examples/quick-start \

0reactions
DieterTimmerscommented, Sep 13, 2022

@tgerdesnv thanks it seems to be working!

Read more comments on GitHub >

github_iconTop Results From Across the Web

[Bug] add_sub example failed · Issue #120 · triton-inference ...
INFO[client.py:82] Model add_sub_i0 load failed: [StatusCode.INTERNAL] failed to load 'add_sub_i0', no version is available It's an INFO, ...
Read more >
Jarvis: Triton server died before reaching ready state ...
I am using Ubuntu 20.04 with RTX 3060, CPU AMD Ryzen 5800X. The error I got is Starting Jarvis Speech Services. This may...
Read more >
HTTP Status Codes List | HTTP Error Codes Explained
Learn about all the HTTP status codes. Read about the HTTP status codes and their descriptions. Quickly understand client and server errors.
Read more >
500 Internal Server Error - HTTP - MDN Web Docs - Mozilla
This error response is a generic "catch-all" response. Usually, this indicates the server cannot find a better 5xx error code to response.
Read more >
HTTP 500 Internal Server Error: What It Means & How to Fix It
This code tells users that the server is temporarily unable to load the page they're looking for. Check out this post for a...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found