Drop Impala UDF function returned NoSuchObjectException error

Impala UDF works a bit differently than Hive UDF, as they are written in different languages. Recently I have encountered an issue that when I try to drop Impala UDF written in C, it failed with below error:

DROP FUNCTION udf_testing.has_vowels;
Query: DROP FUNCTION udf_testing.has_vowels
ERROR:
ImpalaRuntimeException: Error making 'dropFunction' RPC to Hive Metastore:
CAUSED BY: NoSuchObjectException: Function has_vowels does not exist

The “SHOW CREATE FUNCTION” query worked OK as below:

SHOW CREATE FUNCTION has_vowels;
Query: SHOW CREATE FUNCTION has_vowels
+-----------------------------------------------------------------------------+
| result                                                                      |
+-----------------------------------------------------------------------------+
| CREATE FUNCTION udf_testing.has_vowels(STRING)                              |
|  RETURNS BOOLEAN                                                            |
|  LOCATION 'hdfs://nameservice1/user/hive/impala-udf/libudfsample.so'        |
|  SYMBOL='_Z9HasVowelsPN10impala_udf15FunctionContextERKNS_9StringValE'      |
|                                                                             |
+-----------------------------------------------------------------------------+
Fetched 1 row(s) in 0.03s

After researching, it turned out that when dropping functions in Impala, you will also need to specify the function parameters. So below query will work:

DROP FUNCTION udf_testing.has_vowels(STRING);
Query: DROP FUNCTION udf_testing.has_vowels(STRING)

This is not immediately obvious, but it is documented on Cloudera’ offical documentation site: DROP FUNCTION Statement.

Hope above helps.

How to Compile Hive UDF without MVN

Last week I needed to do a quick compile of a sample Hive UDF for testing purpose. I didn’t want to install all Maven stuff and create a pom.xml file just for this simple task. Thanks to my colleague YiBing, who is an expert in Hive, I got it working in just a few commands in the shell.

[root@localhost hive-udf]$ export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera/
[root@localhost hive-udf]$ export PATH=$PATH:$JAVA_HOME/bin
[root@localhost hive-udf]$ javac -cp `hadoop classpath`:"/opt/cloudera/parcels/CDH/lib/hive/lib/*" GenericUDFCurrentUser.java

[root@localhost hive-udf]$ ls
GenericUDFCurrentUser.class GenericUDFCurrentUser.java

[root@localhost hive-udf]$ mkdir -p com/elin
[root@localhost hive-udf]$ mv GenericUDFCurrentUser.class com/elin
[root@localhost hive-udf]$ jar cvf elin-udf.jar com
added manifest
adding: com/(in = 0) (out= 0)(stored 0%)
adding: com/elin/(in = 0) (out= 0)(stored 0%)
adding: com/elin/GenericUDFCurrentUser.class(in = 2505) (out= 1145)(deflated 54%)

Then I could access the UDF in Hive

ADD JAR /user/ericlin/hive-udf/elin-udf.jar;
CREATE TEMPORARY FUNCTION get_user AS com.elin.GenericUDFCurrentUser';
SELECT get_user() from test;

It is so simple without the need of any third party thingy.