Last year, I wrote the first part of my Impala Query Profile series. I have been busy and not able to add the Part 2 of the series, apologies here. If you missed it, please check it out the last post here first: Impala Query Profile Explained – Part 1. …
As we know, Hadoop/HDFS/MapReduce/Impala is designed to store and process large amount of data, in terms of TBs or PBs. And we also know that having too many small files will hurt query performance, because NameNode needs to store millions of metadata to hold the information about files being stored …
If you work with Impala, but have no idea how to interpret the Impala query PROFILEs, it would be very hard to understand what’s going on and how to make your query run at its full potential. I think this is the case for lots of Impala users, so I …
Recently, I discovered a bug in Impala that when you are using Expression in the ORDER BY clause, the query will fail with below error message: Customer used a very complicated query, and I managed to simplify it to look something like below: This can be re-produced from CDH5.13.x onward. …
This article explains Impala daemon’s processes and how to control the maximum memory each process can use. Impala Daemon has two different processes running, one is written in C++, used by backend, mainly for query processing. The other one is written in Java, used by frontend, for query compilations, storing …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!