Flume Collector Sink Decorator Plugin Finished

Before Christmas, I posted a blog about my trouble to write a Flume Collector Sink Decorator Plugin.

After doing some research and continue digging the underlining issue, I finally get a solution to this, which makes me super happy.

The issue I had was caused by error:

Exception in thread "main" java.lang.UnsupportedClassVersionError: garbagefilter/GarbageFilterDecorator : Unsupported major.minor version 51.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:634)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:186)
at com.cloudera.flume.conf.SourceFactoryImpl.loadPluginBuilders(SourceFactoryImpl.java:160)
at com.cloudera.flume.conf.SourceFactoryImpl.(SourceFactoryImpl.java:126)
at com.cloudera.flume.conf.FlumeBuilder.(FlumeBuilder.java:89)
at com.cloudera.flume.agent.LogicalNodeManager.spawn(LogicalNodeManager.java:75)
at com.cloudera.flume.agent.FlumeNode.setup(FlumeNode.java:529)
at com.cloudera.flume.agent.FlumeNode.main(FlumeNode.java:665)

This is due to the fact that I compiled my plugin under Java 1.7 and tried to run the .jar file under Java 1.6. This error did not appear in the flume log when I ran it as a service using command, which was why I had no clue on what was going on:

$ service flume-node start

It only appeared when I ran it directly on the command line:

$ flume node -n collector1

Not exactly sure why though. Anyway, it is now working, simply follow the steps outlined here.

This is my first Java code in the last 5 years, wow. I think I will spend more time to write more Flume plugins as we need to push more jobs to Flume to do some post processing for us.

Leave a Reply

Your email address will not be published. Required fields are marked *