hadoop - Talend HiveDB connection needs cloudera SerDe -
i'm trying connect talend open studio hive.
in hive have table custom defined fields(from cloudera-twitter-example).
open studio find table without problems, if try retrieve schema such error:
and in talend log get:
!entry org.talend.platform.logging 4 0 2015-09-21 10:49:12.375 !message 2015-09-21 10:49:12,372 error org.talend.commons.exception.commonexceptionhandler - java.lang.reflect.invocationtargetexception !stack 0 java.sql.sqlexception: java.lang.reflect.invocationtargetexception @ org.talend.metadata.managment.hive.embeddedhivedatabasemetadata.getcolumns(embeddedhivedatabasemetadata.java:401) @ org.talend.core.model.metadata.builder.database.manager.extractmanager.getcolumnsresultset(extractmanager.java:844) @ org.talend.core.model.metadata.builder.database.manager.extractmanager.extractcolumns(extractmanager.java:641) @ org.talend.core.model.metadata.builder.database.manager.extractmanager.returnmetadatacolumnsformtable(extractmanager.java:521) @ org.talend.core.model.metadata.builder.database.extractmetadatafromdatabase.returnmetadatacolumnsformtable(extractmetadatafromdatabase.java:224) @ org.talend.repository.ui.wizards.metadata.table.database.databasetableform.pressretreiveschemabutton(databasetableform.java:1150) @ org.talend.repository.ui.wizards.metadata.table.database.databasetableform.access$13(databasetableform.java:1121) @ org.talend.repository.ui.wizards.metadata.table.database.databasetableform$4.widgetselected(databasetableform.java:795) @ org.eclipse.swt.widgets.typedlistener.handleevent(typedlistener.java:248) @ org.eclipse.swt.widgets.eventtable.sendevent(eventtable.java:84) @ org.eclipse.swt.widgets.display.sendevent(display.java:4454) @ org.eclipse.swt.widgets.widget.sendevent(widget.java:1388) @ org.eclipse.swt.widgets.display.rundeferredevents(display.java:3799) @ org.eclipse.swt.widgets.display.readanddispatch(display.java:3409) @ org.eclipse.jface.window.window.runeventloop(window.java:832) @ org.eclipse.jface.window.window.open(window.java:808) @ org.talend.repository.metadata.ui.actions.metadata.abstractcreatetableaction.handlewizard(abstractcreatetableaction.java:139) @ org.talend.repository.metadata.ui.actions.metadata.abstractcreatetableaction$1$1.run(abstractcreatetableaction.java:1088) @ org.talend.repository.repositoryworkunit.executerun(repositoryworkunit.java:93) @ org.talend.core.repository.model.abstractrepositoryfactory.executerepositoryworkunit(abstractrepositoryfactory.java:256) @ org.talend.repository.localprovider.model.localrepositoryfactory.executerepositoryworkunit(localrepositoryfactory.java:3227) @ org.talend.core.repository.model.proxyrepositoryfactory.executerepositoryworkunit(proxyrepositoryfactory.java:1996) @ org.talend.repository.metadata.ui.actions.metadata.abstractcreatetableaction$1.runinuithread(abstractcreatetableaction.java:1110) @ org.eclipse.ui.progress.uijob$1.run(uijob.java:97) @ org.eclipse.swt.widgets.runnablelock.run(runnablelock.java:35) @ org.eclipse.swt.widgets.synchronizer.runasyncmessages(synchronizer.java:136) @ org.eclipse.swt.widgets.display.runasyncmessages(display.java:3774) @ org.eclipse.swt.widgets.display.readanddispatch(display.java:3412) @ org.eclipse.e4.ui.internal.workbench.swt.partrenderingengine$9.run(partrenderingengine.java:1151) @ org.eclipse.core.databinding.observable.realm.runwithdefault(realm.java:332) @ org.eclipse.e4.ui.internal.workbench.swt.partrenderingengine.run(partrenderingengine.java:1032) @ org.eclipse.e4.ui.internal.workbench.e4workbench.createandrunui(e4workbench.java:148) @ org.eclipse.ui.internal.workbench$5.run(workbench.java:636) @ org.eclipse.core.databinding.observable.realm.runwithdefault(realm.java:332) @ org.eclipse.ui.internal.workbench.createandrunworkbench(workbench.java:579) @ org.eclipse.ui.platformui.createandrunworkbench(platformui.java:150) @ org.talend.rcp.intro.application.start(application.java:183) @ org.eclipse.equinox.internal.app.eclipseapphandle.run(eclipseapphandle.java:196) @ org.eclipse.core.runtime.internal.adaptor.eclipseapplauncher.runapplication(eclipseapplauncher.java:134) @ org.eclipse.core.runtime.internal.adaptor.eclipseapplauncher.start(eclipseapplauncher.java:104) @ org.eclipse.core.runtime.adaptor.eclipsestarter.run(eclipsestarter.java:380) @ org.eclipse.core.runtime.adaptor.eclipsestarter.run(eclipsestarter.java:235) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:62) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43) @ java.lang.reflect.method.invoke(method.java:497) @ org.eclipse.equinox.launcher.main.invokeframework(main.java:648) @ org.eclipse.equinox.launcher.main.basicrun(main.java:603) @ org.eclipse.equinox.launcher.main.run(main.java:1465) @ org.eclipse.equinox.launcher.main.main(main.java:1438) caused by: java.lang.reflect.invocationtargetexception @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:62) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43) @ java.lang.reflect.method.invoke(method.java:497) @ org.talend.metadata.managment.hive.embeddedhivedatabasemetadata.getcolumns(embeddedhivedatabasemetadata.java:370) ... 49 more caused by: java.lang.runtimeexception: metaexception(message:java.lang.classnotfoundexception class com.cloudera.hive.serde.jsonserde not found) @ org.apache.hadoop.hive.ql.metadata.table.getdeserializerfrommetastore(table.java:276) @ org.apache.hadoop.hive.ql.metadata.table.getdeserializer(table.java:256) @ org.apache.hadoop.hive.ql.metadata.table.getcols(table.java:595) @ org.apache.hadoop.hive.ql.metadata.table.getallcols(table.java:612) ... 54 more caused by: metaexception(message:java.lang.classnotfoundexception class com.cloudera.hive.serde.jsonserde not found) @ org.apache.hadoop.hive.metastore.metastoreutils.getdeserializer(metastoreutils.java:385) @ org.apache.hadoop.hive.ql.metadata.table.getdeserializerfrommetastore(table.java:274) ... 57 more
most important message
caused by: java.lang.runtimeexception: metaexception(message:java.lang.classnotfoundexception class com.cloudera.hive.serde.jsonserde not found) @ org.apache.hadoop.hive.ql.metadata.table.getdeserializerfrommetastore(table.java:276) @ org.apache.hadoop.hive.ql.metadata.table.getdeserializer(table.java:256) @ org.apache.hadoop.hive.ql.metadata.table.getcols(table.java:595) @ org.apache.hadoop.hive.ql.metadata.table.getallcols(table.java:612) ... 54 more caused by: metaexception(message:java.lang.classnotfoundexception class com.cloudera.hive.serde.jsonserde not found) @ org.apache.hadoop.hive.metastore.metastoreutils.getdeserializer(metastoreutils.java:385) @ org.apache.hadoop.hive.ql.metadata.table.getdeserializerfrommetastore(table.java:274) ... 57 more
this message shows talend open studio big data cannot find serde.
placed hive-serede on hadoop cluster knots , informatica powercenter can schema information hive.
when use hive cli on knots or hue run queries on hive works without problems , configurations
add jar
how can make talend open studio working hive tables?
- hadoop: cdh 5.4.3
- talend open studio 6.0.1
- hive 1.1.0+cdh5.4.3+151
i solved issue. default talend open studio trying work hive metastore directly.
called embedded connection. (metastore port 9093)
in connection setting saw port 10000 pointed me hiveserver2(thrift).
after switching standalone connection in began working.
Comments
Post a Comment