java - ClassCastException while using HFileOutputFormat2 -

-

i trying upload data file in hdfs hbase table using hfileoutputformat2 outputformat getting following exception, 

java.lang.exception: java.lang.classcastexception: org.apache.hadoop.hbase.client.put cannot cast org.apache.hadoop.hbase.cell @ org.apache.hadoop.mapred.localjobrunner$job.runtasks(localjobrunner.java:462) @ org.apache.hadoop.mapred.localjobrunner$job.run(localjobrunner.java:522)  caused by: java.lang.classcastexception: org.apache.hadoop.hbase.client.put cannot cast org.apache.hadoop.hbase.cell @ org.apache.hadoop.hbase.mapreduce.hfileoutputformat2$1.write(hfileoutputformat2.java:148) @ org.apache.hadoop.mapred.maptask$newdirectoutputcollector.write(maptask.java:635) @ org.apache.hadoop.mapreduce.task.taskinputoutputcontextimpl.write(taskinputoutputcontextimpl.java:89) @ org.apache.hadoop.mapreduce.lib.map.wrappedmapper$context.write(wrappedmapper.java:112) @ com.xogito.ingestion.mr.hbase.cseventshbasemapper.map(cseventshbasemapper.java:90) @ com.xogito.ingestion.mr.hbase.cseventshbasemapper.map(cseventshbasemapper.java:1) @ org.apache.hadoop.mapreduce.mapper.run(mapper.java:145) @ org.apache.hadoop.mapred.maptask.runnewmapper(maptask.java:764) @ org.apache.hadoop.mapred.maptask.run(maptask.java:340) @ org.apache.hadoop.mapred.localjobrunner$job$maptaskrunnable.run(localjobrunner.java:243) @ java.util.concurrent.executors$runnableadapter.call(executors.java:441) @ java.util.concurrent.futuretask$sync.innerrun(futuretask.java:303) @ java.util.concurrent.futuretask.run(futuretask.java:138) @ java.util.concurrent.threadpoolexecutor$worker.runtask(threadpoolexecutor.java:886) @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:908) @ java.lang.thread.run(thread.java:662)

following code of job

    @override     public int run(string[] args) throws exception {     configuration conf = getconf();     job = job.getinstance(conf, "my_job");     job.setjarbyclass(getclass());     job.setmapoutputkeyclass(immutablebyteswritable.class);     job.setmapoutputvalueclass(put.class);     job.setspeculativeexecution(false);       job.setreducespeculativeexecution(false);       job.setmapperclass(custommapper.class);//custom mapper     job.setinputformatclass(textinputformat.class);     job.setoutputformatclass(hfileoutputformat.class);       string parentinputpath = args[0];     string parentoutputpath = args[1];      fileinputformat.addinputpaths(job, inputpath);     hfileoutputformat.setoutputpath(job,new path(parentoutputpath));      configuration hconf = hbaseconfiguration.create(conf);     hconf.set("hbase.zookeeper.quorum", "x.x.x.x");     hconf.set("hbase.zookeeper.property.clientport", "2181");     htable htable = new htable(hconf, "mytable");    // htable.setautoflush(false, true);   //  htable.setwritebuffersize(1024 * 1024 * 12);     hfileoutputformat.configureincrementalload(job, htable);     job.setnumreducetasks(0);     job.submit();   }

and mapper's code following,

  @override public void map(writablecomparable key, writable  val, context context) throws ioexception, interruptedexception{     string data = val.tostring();     string[] splitted = data.split("\t");     string account = splitted[1];     matcher match = account.matcher(account);     int clientid = 0;     if (match.find()) {         clientid = integer.valueof(integer.parseint(match                 .group(1)));     }     string userid = splitted[2];     long timestamp = 0l;     try {         timestamp = long.valueof(splitted[10]);     } catch (exception e) {         logger.error(e.getmessage(), e);     }      string rowkeytext = "somtext";     immutablebyteswritable rowkey = new           immutablebyteswritable(bytes.tobytes(rowkeytext));      put put = new put(bytes.tobytes(rowkeytext));     put.add(cf,column, value);      context.write(rowkey, put);    }

hfileoutputformat or new version hfileoutputformat2 needs keyvalue final class. maybe putsortreducer not correctly applied converts put keyvalue instances.

in case not using mapreduce spark, creating keyvalue's directly instead of put's


Comments

Post a Comment

Popular posts from this blog

javascript - ScrollTo Effect (href to div) -

-
i can't seem find working simple scrollto effect. want basic, it's when click on link in nav bar, must take me selected div id scrollto effect. this code have. <nav> <ul> <li><a href="#home">home</a></li> <li><a href="#about">about</a></li> <li><a href="#contact">contact</a></li> </ul> </nav> <section id="home"> content </section> <section id="about"> content </section> <section id="contact"> content </section> js: <!----- navegación slide ---> $(document).ready(function() {   $('a[href*=#]').each(function() {     if (location.pathname.replace(/^\//,'') == this.pathname.replace(/^\//,'')     && location.hostname == this.hostname     && this.hash.replace(/#/,'') ) {       var $targetid = $(this.hash), $targetancho...
Read more

javascript - ng-class not responding to change in model in Angular? -

-
having searched extensively, got a ternary in templates , getting partially functional results, i'm 99% sure i'm doing 99% right, but...obviously not completely, i'm here. the basic html structure: <ul> <li class="week-row" ng-repeat="week in weeks" id="{{$index}}"> <ul class="tiles" ng-class="{ 'expanded': week.isopen, 'hidden': !week.isopen }"> <li class="day-tile" ng-class="{'active': (activeday == $parent.$index + '-' + $index) }" id="{{$parent.$index + '-' + $index}}" ng-repeat="day in week.days" ng-click="fn2({ week: $parent.$index, day: $index })"> <!-- moved ng-class here, , works? --> <div>some stuff in here</div> </li> </ul...
Read more

javascript - document.registerElement extending HTMLInputElement prototype while using custom tag name to avoid implicit browser style -

-
my goal two-fold: display html element keeps track of 'checked' state , triggers 'change' events without having manually implement code myself. not fight browser's implicit styling of element -webkit-appearance: none , more fussing. i need support current, stable channel of google chrome. such, not have concern compatibility non-modern browsers. i've approached problem using document.registerelement . method, register new tag, custom-checkbox , , inform browser tag inherit input. from there, need setup fact input's type checkbox. so, set type in createdcallback method ran when html element created. this code has issues: it works when not use custom tag , instead leverage is property inform browser input 'is' custom-checkbox . however, using <input> tag causes browser take on styling element. not wish have happen. when using custom tag, createdcallback method not ran. such, element not setup fact checkbox , not fire ...
Read more