QnaList > Groups > Sqoop-User > Oct 2013
faq

Programmatic (java) Sqoop Import From MySQL To HBase Could Not Find Hadoop Core Jar

Hi all,
I'm new to Sqoop and I'm trying to execute (from Java code) a sqoop import
from MySQL to HBase.
I'm using Sqoop.runTool() with a String[] containing all the necessary
parameters, I even added "--hadoop-home" with the value of my $HADOOP_HOME.
Anyone had this problem before ? any ideas ?
Thanks,
Amit.

asked Oct 27 2013 at 07:04

Amit Sela 's gravatar image



8 Replies for : Programmatic (java) Sqoop Import From MySQL To HBase Could Not Find Hadoop Core Jar
Can you try adding the jar to HADOOP_CLASSPATH / CLASSPATH ?

answered Oct 27 2013 at 08:15

Krishnan K 's gravatar image


Which jar ?
Hadoop jars are in "HADOOP_HOME/lib" and also sqoop jar.
I'm trying to execute the import from an OSGi running client, with a maven
dependency and embedding on sqoop.

answered Oct 27 2013 at 08:52

Amit Sela 's gravatar image


Hi Amit,
would you mind sharing the entire stack trace that you are getting?
Please note that Sqoop 1 do not have officially supported client API and thus directly using Sqoop.runTool() is not supported and you might run into a lot of issues such as this one.
Jarcec

answered Oct 30 2013 at 14:14

Jarek Jarcec Cecho 's gravatar image


That's the stack trace for the exception I got:
Encountered IOException running import job: java.io.IOException: Could not
find hadoop core jar!
at
org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:146)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at
com.infolinks.redmap.services.impl.utils.Sqooper.importMysqlToHBase(Sqooper.java:35)
at
com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:79)
at
com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:76)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
My infrastructure allows me to execute MR jobs and to access HBase so I
don't understand why executing Sqoop results with "Could not find hadoop
core jar!" unless Sqoop is actually looking for the jar to deploy it for
some reason...

answered Oct 31 2013 at 02:09

Amit Sela 's gravatar image


Hi Amit,
what Hadoop version are you using? Would you mind sharing with us listing of the directory that you specified in the --hadoop-home argument?
Please note that Sqoop 1 do not have officially supported client API and thus directly using Sqoop.runTool() is not supported and you might run into a lot of issues such as this one.
Jarcec

answered Nov 1 2013 at 18:29

Jarek Jarcec Cecho 's gravatar image


I'm running Hadoop 1.0.4 (and HBAse 0.94.2) and the directory I specify for
That's the exact definition in our system.

answered Nov 3 2013 at 05:06

Amit Sela 's gravatar image


Hi Amit,
the parameter --hadoop-home should be pointing to $HADOOP_HOME, not to a $HADOOP_HOME/bin.
Jarcec

answered Nov 3 2013 at 13:04

Jarek Jarcec Cecho 's gravatar image


It points to $HADOOP_HOME/bin/.. Meaning $HADOOP_HOME.
I know it looks stupid to write it that way but I wanted it to be a
complete match to the definition as running from command line.
I'll give a try the normal way.

answered Nov 3 2013 at 15:18

Amit Sela 's gravatar image


Related discussions

Tagged

Group Sqoop-user

asked Oct 27 2013 at 07:04

active Nov 3 2013 at 15:18

posts:9

users:3

©2013 QnaList.com . QnaList is part of ZisaTechnologies LLC.