【原创】大叔经验分享（48）oozie中通过shell执行impala

oozie中通过shell执行impala，脚本如下：

$ cat test_impala.sh

#!/bin/sh

/usr/bin/kinit -kt /tmp/impala.keytab impala/server04

/usr/bin/impala-shell -i server04:21000 -q 'show databases'

直接执行shell脚本正常，在oozie中执行报错：

Traceback (most recent call last):

  File "/usr/lib/impala-shell/impala_shell.py", line 38, in <module>

    from impala_client import (ImpalaClient, DisconnectedException, QueryStateException,

  File "/usr/lib/impala-shell/lib/impala_client.py", line 20, in <module>

    import sasl

  File "/usr/lib/impala-shell/ext-py/sasl-0.1.1-py2.7-linux-x86_64.egg/sasl/__init__.py", line 1, in <module>

  File "/usr/lib/impala-shell/ext-py/sasl-0.1.1-py2.7-linux-x86_64.egg/sasl/saslwrapper.py", line 7, in <module>

  File "/usr/lib/impala-shell/ext-py/sasl-0.1.1-py2.7-linux-x86_64.egg/_saslwrapper.py", line 7, in <module>

  File "/usr/lib/impala-shell/ext-py/sasl-0.1.1-py2.7-linux-x86_64.egg/_saslwrapper.py", line 4, in __bootstrap__

  File "/usr/lib/impala-shell/lib/pkg_resources.py", line 874, in resource_filename

    self, resource_name

  File "/usr/lib/impala-shell/lib/pkg_resources.py", line 1349, in get_resource_filename

    self._extract_resource(manager, self._eager_to_zip(name))

  File "/usr/lib/impala-shell/lib/pkg_resources.py", line 1375, in _extract_resource

    self.egg_name, self._parts(zip_path)

  File "/usr/lib/impala-shell/lib/pkg_resources.py", line 954, in get_cache_path

    self.extraction_error()

  File "/usr/lib/impala-shell/lib/pkg_resources.py", line 920, in extraction_error

    raise err

pkg_resources.ExtractionError: Can't extract file(s) to egg cache

The following error occurred while trying to extract file(s) to the Python egg

cache:

  [Errno 13] Permission denied: '/home/.python-eggs'

The Python egg cache directory is currently set to:

  /home/.python-eggs

Perhaps your account does not have write access to this directory?  You can

change the cache directory by setting the PYTHON_EGG_CACHE environment

variable to point to an accessible directory.

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]

按照提示修改PYTHON_EGG_CACHE即可

$ cat test_impala.sh

#!/bin/sh

export PYTHON_EGG_CACHE=/tmp/.python-eggs

/usr/bin/kinit -kt /tmp/impala.keytab impala/server04

/usr/bin/impala-shell -i server04:21000 -q 'show databases'

另外还可以通过beeline连接，则没有python的问题，

详见：https://www.cnblogs.com/barneywill/p/10296502.html

巴特西

【原创】大叔经验分享（48）oozie中通过shell执行impala

最新文章

热门文章