helm install airflow in namespace get error: File "<string>", line 32, in <module> TimeoutError: There are still unapplied migrations after 60 seconds

See original GitHub issue

Apache Airflow version: master git

Kubernetes version (if you are using kubernetes) (use kubectl version):


Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.17", 

Environment:

  • Cloud provider or hardware configuration: Microk8s
  • OS (e.g. from /etc/os-release): VERSION=“18.04.3 LTS (Bionic Beaver)”

What happened:

git clone https://github.com/apache/airflow.git
cd airflow/chart/
helm dependency update
kubectl create namespace xxxxx
werf helm install --wait --set webserver.defaultUser.password=password,ingress.enabled=true,ingress.hosts[0]=airflow.192.168.22.7.xip.io --namespace xxxxx airflow ./

Log

│ ┌ deploy/airflow-webserver po/airflow-webserver-86857b5969-sqkv6 container/wait-for-airflow-migrations logs
│ │ [2021-04-13 05:57:20,571] {<string>:35} INFO - Waiting for migrations... 60 second(s)
│ │ Traceback (most recent call last):
│ │   File "<string>", line 32, in <module>
│ │ TimeoutError: There are still unapplied migrations after 60 seconds.
│ └ deploy/airflow-webserver po/airflow-webserver-86857b5969-sqkv6 container/wait-for-airflow-migrations logs

Next line log

│ deploy/airflow-scheduler ERROR: po/airflow-scheduler-658d5d4454-r2sgl container/wait-for-airflow-migrations: CrashLoopBackOff: back-off 10s restarting failed container=wait-for-airflow-migrations             ↵
│ pod=airflow-scheduler-658d5d4454-r2sgl_sdpcc(40e85057-2aa5-4e9e-a47d-e91530038c0c)
│ 1/1 allowed errors occurred for deploy/airflow-scheduler: continue tracking

Full log https://gist.github.com/patsevanton/0edd5571cf69aa539edcdb803c288061

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:2
  • Comments:36 (7 by maintainers)

github_iconTop GitHub Comments

6reactions
autarchprincepscommented, Aug 11, 2021

I’m facing the same issue. I don’t ever get any pod or job containing run-airflow-migrations, and consequently the wait never ends. Is there a solution for this? I’m not running terraform and neither is patsevanto. He is using werf, I’m using flux.

4reactions
dingobarcommented, Aug 4, 2022

I came to this issue upgrading to airflow 2.3.0 via helm chart 1.6.0. The issue turned out to be the migration job not being able to schedule due to a too low CPU request limit on the k8s namespace. Just another thing to check if you end up here like I did.

Read more comments on GitHub >

github_iconTop Results From Across the Web

helm install airflow in namespace get error ... - The Mail Archive
... helm install airflow in namespace get error: File " ", line 32, in TimeoutError: There are still unapplied migrations after 60 seconds....
Read more >
Run-airflow-migration and wait-for-airflow-migrations
I am installing Astronomer Airflow using the Helm chart and have a question ... There are still unapplied migrations after 60 seconds.
Read more >
Airflow deployment stuck after using config from Vault backend
60 second(s) Traceback (most recent call last): File "<string>", line 46, in <module> TimeoutError: There are still unapplied migrations after ...
Read more >
Helm Chart for Apache Airflow
This chart will bootstrap an Airflow deployment on a Kubernetes cluster using ... helm upgrade --install airflow apache-airflow/airflow --namespace airflow ...
Read more >
Airflow Helm Chart (User Community) - Artifact Hub
This chart provides a standard way to deploy Apache Airflow on your Kubernetes cluster, and is used by thousands of companies for their...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found