Symbolic link structure is: ln -s #PATHDIRECTED #LINKNAME
ln -s /home/airflow/.local/lib/python2.7 python2. To create a symbolic link so that /home/airflow/.local/lib/python2.7/ is used, do the following and create the following symbolic links: I am going to assume python and which python leads to python 3 (which you do not want to use), but python2 and python2.7 lead to the instance you do want to use. You can also do which pip to see what pip instance is being used. Or type which python to find which python instance is being used by your vm. Type the commands and see which instance of python is used (2.7.5, 2.7.14, 3.6, etc): This can be solved easily by using symbolic links to redirect to 2.7. There is a pip with python3.6 that is trying to be used, but all of your modules are actually with your python 2.7. It looks like your python environment is degraded - you have multiple instances of python on your vm (python 3.6 and python 2.7) and multiple instances of pip. PATH=/usr/local/airflow/instances/airflow2/venv/bin:/usr/local/bin:/usr/bin:/binĪIRFLOW_HOME=/usr/local/airflow/instances/airflow2/homeĪIRFLOW_CONFIG=/usr/local/airflow/instances/airflow2/etc/airflow.cfg This is the EnvironentFile Contents uses from above more /usr/local/airflow/instances/airflow2/etc/envars ::::::::::::::ĮnvironmentFile=/usr/local/airflow/instances/airflow2/etc/envarsĮxecStart=/usr/local/airflow/instances/airflow2/venv/bin/airflow schedulerĮxecStart=/usr/local/airflow/instances/airflow2/venv/bin/airflow webserver What's the correct way to go about this?īelow is my systemctl scripts. bashrc but doesn't seem to be read when the dag jobs are executed. I've tried exporting PYTHONPATH in the Airflow user accounts. #Airflow docker emr code
That doesn't feel right though having to set my path in my code directly. ('/home/airflow/.local/lib/python2.7/site-packages') Inside of a file I can directly modify the python sys.path and that seems to mitigate my issue. Inside of the Airflow UI, I see a ton of these message variations.īroken DAG: No module named 'paramiko' I have some dags that can't seem to locate python modules.
0 kommentar(er)
