hadoop - Talend HiveDB connection needs cloudera SerDe -


i'm trying connect talend open studio hive.

in hive have table custom defined fields(from cloudera-twitter-example).
open studio find table without problems, if try retrieve schema such error: error message

and in talend log get:

!entry org.talend.platform.logging 4 0 2015-09-21 10:49:12.375 !message 2015-09-21 10:49:12,372 error org.talend.commons.exception.commonexceptionhandler  - java.lang.reflect.invocationtargetexception  !stack 0 java.sql.sqlexception: java.lang.reflect.invocationtargetexception     @ org.talend.metadata.managment.hive.embeddedhivedatabasemetadata.getcolumns(embeddedhivedatabasemetadata.java:401)     @ org.talend.core.model.metadata.builder.database.manager.extractmanager.getcolumnsresultset(extractmanager.java:844)     @ org.talend.core.model.metadata.builder.database.manager.extractmanager.extractcolumns(extractmanager.java:641)     @ org.talend.core.model.metadata.builder.database.manager.extractmanager.returnmetadatacolumnsformtable(extractmanager.java:521)     @ org.talend.core.model.metadata.builder.database.extractmetadatafromdatabase.returnmetadatacolumnsformtable(extractmetadatafromdatabase.java:224)     @ org.talend.repository.ui.wizards.metadata.table.database.databasetableform.pressretreiveschemabutton(databasetableform.java:1150)     @ org.talend.repository.ui.wizards.metadata.table.database.databasetableform.access$13(databasetableform.java:1121)     @ org.talend.repository.ui.wizards.metadata.table.database.databasetableform$4.widgetselected(databasetableform.java:795)     @ org.eclipse.swt.widgets.typedlistener.handleevent(typedlistener.java:248)     @ org.eclipse.swt.widgets.eventtable.sendevent(eventtable.java:84)     @ org.eclipse.swt.widgets.display.sendevent(display.java:4454)     @ org.eclipse.swt.widgets.widget.sendevent(widget.java:1388)     @ org.eclipse.swt.widgets.display.rundeferredevents(display.java:3799)     @ org.eclipse.swt.widgets.display.readanddispatch(display.java:3409)     @ org.eclipse.jface.window.window.runeventloop(window.java:832)     @ org.eclipse.jface.window.window.open(window.java:808)     @ org.talend.repository.metadata.ui.actions.metadata.abstractcreatetableaction.handlewizard(abstractcreatetableaction.java:139)     @ org.talend.repository.metadata.ui.actions.metadata.abstractcreatetableaction$1$1.run(abstractcreatetableaction.java:1088)     @ org.talend.repository.repositoryworkunit.executerun(repositoryworkunit.java:93)     @ org.talend.core.repository.model.abstractrepositoryfactory.executerepositoryworkunit(abstractrepositoryfactory.java:256)     @ org.talend.repository.localprovider.model.localrepositoryfactory.executerepositoryworkunit(localrepositoryfactory.java:3227)     @ org.talend.core.repository.model.proxyrepositoryfactory.executerepositoryworkunit(proxyrepositoryfactory.java:1996)     @ org.talend.repository.metadata.ui.actions.metadata.abstractcreatetableaction$1.runinuithread(abstractcreatetableaction.java:1110)     @ org.eclipse.ui.progress.uijob$1.run(uijob.java:97)     @ org.eclipse.swt.widgets.runnablelock.run(runnablelock.java:35)     @ org.eclipse.swt.widgets.synchronizer.runasyncmessages(synchronizer.java:136)     @ org.eclipse.swt.widgets.display.runasyncmessages(display.java:3774)     @ org.eclipse.swt.widgets.display.readanddispatch(display.java:3412)     @ org.eclipse.e4.ui.internal.workbench.swt.partrenderingengine$9.run(partrenderingengine.java:1151)     @ org.eclipse.core.databinding.observable.realm.runwithdefault(realm.java:332)     @ org.eclipse.e4.ui.internal.workbench.swt.partrenderingengine.run(partrenderingengine.java:1032)     @ org.eclipse.e4.ui.internal.workbench.e4workbench.createandrunui(e4workbench.java:148)     @ org.eclipse.ui.internal.workbench$5.run(workbench.java:636)     @ org.eclipse.core.databinding.observable.realm.runwithdefault(realm.java:332)     @ org.eclipse.ui.internal.workbench.createandrunworkbench(workbench.java:579)     @ org.eclipse.ui.platformui.createandrunworkbench(platformui.java:150)     @ org.talend.rcp.intro.application.start(application.java:183)     @ org.eclipse.equinox.internal.app.eclipseapphandle.run(eclipseapphandle.java:196)     @ org.eclipse.core.runtime.internal.adaptor.eclipseapplauncher.runapplication(eclipseapplauncher.java:134)     @ org.eclipse.core.runtime.internal.adaptor.eclipseapplauncher.start(eclipseapplauncher.java:104)     @ org.eclipse.core.runtime.adaptor.eclipsestarter.run(eclipsestarter.java:380)     @ org.eclipse.core.runtime.adaptor.eclipsestarter.run(eclipsestarter.java:235)     @ sun.reflect.nativemethodaccessorimpl.invoke0(native method)     @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:62)     @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43)     @ java.lang.reflect.method.invoke(method.java:497)     @ org.eclipse.equinox.launcher.main.invokeframework(main.java:648)     @ org.eclipse.equinox.launcher.main.basicrun(main.java:603)     @ org.eclipse.equinox.launcher.main.run(main.java:1465)     @ org.eclipse.equinox.launcher.main.main(main.java:1438) caused by: java.lang.reflect.invocationtargetexception     @ sun.reflect.nativemethodaccessorimpl.invoke0(native method)     @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:62)     @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43)     @ java.lang.reflect.method.invoke(method.java:497)     @ org.talend.metadata.managment.hive.embeddedhivedatabasemetadata.getcolumns(embeddedhivedatabasemetadata.java:370)     ... 49 more caused by: java.lang.runtimeexception: metaexception(message:java.lang.classnotfoundexception class com.cloudera.hive.serde.jsonserde not found)     @ org.apache.hadoop.hive.ql.metadata.table.getdeserializerfrommetastore(table.java:276)     @ org.apache.hadoop.hive.ql.metadata.table.getdeserializer(table.java:256)     @ org.apache.hadoop.hive.ql.metadata.table.getcols(table.java:595)     @ org.apache.hadoop.hive.ql.metadata.table.getallcols(table.java:612)     ... 54 more caused by: metaexception(message:java.lang.classnotfoundexception class com.cloudera.hive.serde.jsonserde not found)     @ org.apache.hadoop.hive.metastore.metastoreutils.getdeserializer(metastoreutils.java:385)     @ org.apache.hadoop.hive.ql.metadata.table.getdeserializerfrommetastore(table.java:274)     ... 57 more 

most important message

caused by: java.lang.runtimeexception: metaexception(message:java.lang.classnotfoundexception class com.cloudera.hive.serde.jsonserde not found) @ org.apache.hadoop.hive.ql.metadata.table.getdeserializerfrommetastore(table.java:276) @ org.apache.hadoop.hive.ql.metadata.table.getdeserializer(table.java:256) @ org.apache.hadoop.hive.ql.metadata.table.getcols(table.java:595) @ org.apache.hadoop.hive.ql.metadata.table.getallcols(table.java:612) ... 54 more caused by: metaexception(message:java.lang.classnotfoundexception class com.cloudera.hive.serde.jsonserde not found) @ org.apache.hadoop.hive.metastore.metastoreutils.getdeserializer(metastoreutils.java:385) @ org.apache.hadoop.hive.ql.metadata.table.getdeserializerfrommetastore(table.java:274) ... 57 more 

this message shows talend open studio big data cannot find serde.
placed hive-serede on hadoop cluster knots , informatica powercenter can schema information hive.
when use hive cli on knots or hue run queries on hive works without problems , configurations

add jar

how can make talend open studio working hive tables?

  • hadoop: cdh 5.4.3
  • talend open studio 6.0.1
  • hive 1.1.0+cdh5.4.3+151

i solved issue. default talend open studio trying work hive metastore directly.
called embedded connection. (metastore port 9093)
in connection setting saw port 10000 pointed me hiveserver2(thrift).
after switching standalone connection in began working.


Comments

Popular posts from this blog

java - Date formats difference between yyyy-MM-dd'T'HH:mm:ss and yyyy-MM-dd'T'HH:mm:ssXXX -

c# - Get rid of xmlns attribute when adding node to existing xml -