Getting class is not whitelisted when I'm using on databricks

See original GitHub issue

Hi, I have installed the spark-nlp on databricks cluster. i was able to import it but when i try to use it it is throwing the error

from sparknlp.pretrained import PretrainedPipeline

explain_document_pipeline = PretrainedPipeline("explain_document_ml")
annotations = explain_document_pipeline.annotate("We are very happy about SparkNLP")
print(annotations)

py4j.security.Py4JSecurityException: Method public static java.lang.String com.johnsnowlabs.nlp.pretrained.PythonResourceDownloader.getDownloadSize(java.lang.String,java.lang.String,java.lang.String) is not whitelisted on class class com.johnsnowlabs.nlp.pretrained.PythonResourceDownloader

Any help is appreciated.

Issue Analytics

State:
Created 2 years ago
Reactions:1
Comments:5 (2 by maintainers)

Top GitHub Comments

1reaction

maziyarpanahicommented, Mar 9, 2022

You didn’t share the screenshots I asked for, so I go with what I can say. This happens in Databricks under certain situations:

Problem:

you have some configs (or those are automatically set by that runtime) that conflict with HC cluster mode

Solution:

Either choose another Cluster mode. (not high concurrency)
or follow this instruction given by Databricks (setting only one of them to true and the other one to false should fix the issue): https://kb.databricks.com/python/python-cmd-fail-high-con-cluster.html

0reactions

github-actions[bot]commented, Aug 6, 2022

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 5 days