Segmentation fault (core dumped) after importing transformers

See original GitHub issue

🐛 Bug

Information

Model I am using (Bert, XLNet …): Bert

Language I am using the model on (English, Chinese …): English

The problem arises when using:

  • the official example scripts: (give details below)
  • my own modified scripts: (give details below)

The tasks I am working on is:

  • an official GLUE/SQUaD task: (give the name)
  • my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

1.import torch 2.import transformers 3.torch.rand(4,5)

import torch
import transformers
torch.rand(4,5)

Expected behavior

I got an error ‘segmentation fault (core dumped)’ while trying to generate a tensor after importing transformers, but if I removed ‘import transformers’, the tensor could be generated. image

Environment info

  • transformers version: 2.11.0
  • Platform: Ubuntu 18.04.4 LTS
  • Python version: 3.7.6
  • PyTorch version (GPU?): 1.4.0+cu100
  • Tensorflow version (GPU?): 2.2.0
  • Using GPU in script?: No
  • Using distributed or parallel set-up in script?: No

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

9reactions
LysandreJikcommented, Jun 29, 2020
5reactions
huangxiaoshuocommented, Jun 30, 2020

Hi, I believe this error may be due to the sentencepiece version 0.1.92 which causes a segmentation fault..

Thank you very much, this error disappeared when sentencpiece was downgraded to 0.1.91. BTW, is there some debugging method to find this root cause when we encounter segmentation fault?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Segmentation fault (Core dumped) with datasets - Intermediate
While trying to download a large dataset(~100GB), without streaming mode like this: from datasets import load_dataset mc4_dataset ...
Read more >
Segmentation fault (core dump) while training a Hugging Face ...
I am trying to train a Machine Translation model from HuggingFace (t5-large) using the europarl_bilingual dataset.
Read more >
Segmentation Fault - Notebook - Jupyter Community Forum
While running programs in jupyter notebook for a long time, I get segmentation fault (core dumped) · When a large dataset is getting...
Read more >
Seg Fault with Pytorch Lightning
I'm running a script with pytorch lightning and keep getting the below Segmentation Fault error. I really have no idea what's going on/how ......
Read more >
引入transformers 报错Segmentation fault (core dumped) - 博客园
其中 transformers 为pip install 自动安装的最新版。 ... 原因:. transformers安装时的一个依赖库sentencepiece出了问题,安装的版本是0.1.92,改回上一 ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found