Today I was trying to index LZO file using hadoop command: However, it failed with the following error: After some research, it turned out that I did not have LZO codec enabled in my cluster. I did the following to resolve the issue: – Add gplextras parcel to Cloudera Manager’s …
Apache Sqoop became the Top-Level Project in Apache in March 2012. Since then, Sqoop has developed a lot and become very popular amongst Hadoop ecosystem. In this post, I will cover the ways to specify database passwords to Sqoop in a secure way. The following ways are common to pass …
When Hive tries to “INSERT OVERWRITE” to a partition of an external table under existing directory, depending on whether the partition definition already exists in the metastore or not, Hive will behave differently: 1) if partition definition does not exist, it will not try to guess where the target partition …
I had some drama yesterday when I was trying to setup my laptop for Cloudera’s Hadoop Administrator Exam with Innovative Exams (http://examslocal.com/). I was unable to share my screen inside Chrome with the examiner. I spent about 20 minutes trying different ways and finally it worked, but I had no …
Shark/Spark will not be able to read Snappy compressed data out of the box. In my previous post, I explained how to enable Snappy compression in Hadoop 2.4, once this is done, enabling Snappy in Spark is dead simple, all you need to do is to set an envionment variable …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!