java - ClassCastException while using HFileOutputFormat2 -

-

i trying upload data file in hdfs hbase table using hfileoutputformat2 outputformat getting following exception, 

java.lang.exception: java.lang.classcastexception: org.apache.hadoop.hbase.client.put cannot cast org.apache.hadoop.hbase.cell @ org.apache.hadoop.mapred.localjobrunner$job.runtasks(localjobrunner.java:462) @ org.apache.hadoop.mapred.localjobrunner$job.run(localjobrunner.java:522)  caused by: java.lang.classcastexception: org.apache.hadoop.hbase.client.put cannot cast org.apache.hadoop.hbase.cell @ org.apache.hadoop.hbase.mapreduce.hfileoutputformat2$1.write(hfileoutputformat2.java:148) @ org.apache.hadoop.mapred.maptask$newdirectoutputcollector.write(maptask.java:635) @ org.apache.hadoop.mapreduce.task.taskinputoutputcontextimpl.write(taskinputoutputcontextimpl.java:89) @ org.apache.hadoop.mapreduce.lib.map.wrappedmapper$context.write(wrappedmapper.java:112) @ com.xogito.ingestion.mr.hbase.cseventshbasemapper.map(cseventshbasemapper.java:90) @ com.xogito.ingestion.mr.hbase.cseventshbasemapper.map(cseventshbasemapper.java:1) @ org.apache.hadoop.mapreduce.mapper.run(mapper.java:145) @ org.apache.hadoop.mapred.maptask.runnewmapper(maptask.java:764) @ org.apache.hadoop.mapred.maptask.run(maptask.java:340) @ org.apache.hadoop.mapred.localjobrunner$job$maptaskrunnable.run(localjobrunner.java:243) @ java.util.concurrent.executors$runnableadapter.call(executors.java:441) @ java.util.concurrent.futuretask$sync.innerrun(futuretask.java:303) @ java.util.concurrent.futuretask.run(futuretask.java:138) @ java.util.concurrent.threadpoolexecutor$worker.runtask(threadpoolexecutor.java:886) @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:908) @ java.lang.thread.run(thread.java:662)

following code of job

    @override     public int run(string[] args) throws exception {     configuration conf = getconf();     job = job.getinstance(conf, "my_job");     job.setjarbyclass(getclass());     job.setmapoutputkeyclass(immutablebyteswritable.class);     job.setmapoutputvalueclass(put.class);     job.setspeculativeexecution(false);       job.setreducespeculativeexecution(false);       job.setmapperclass(custommapper.class);//custom mapper     job.setinputformatclass(textinputformat.class);     job.setoutputformatclass(hfileoutputformat.class);       string parentinputpath = args[0];     string parentoutputpath = args[1];      fileinputformat.addinputpaths(job, inputpath);     hfileoutputformat.setoutputpath(job,new path(parentoutputpath));      configuration hconf = hbaseconfiguration.create(conf);     hconf.set("hbase.zookeeper.quorum", "x.x.x.x");     hconf.set("hbase.zookeeper.property.clientport", "2181");     htable htable = new htable(hconf, "mytable");    // htable.setautoflush(false, true);   //  htable.setwritebuffersize(1024 * 1024 * 12);     hfileoutputformat.configureincrementalload(job, htable);     job.setnumreducetasks(0);     job.submit();   }

and mapper's code following,

  @override public void map(writablecomparable key, writable  val, context context) throws ioexception, interruptedexception{     string data = val.tostring();     string[] splitted = data.split("\t");     string account = splitted[1];     matcher match = account.matcher(account);     int clientid = 0;     if (match.find()) {         clientid = integer.valueof(integer.parseint(match                 .group(1)));     }     string userid = splitted[2];     long timestamp = 0l;     try {         timestamp = long.valueof(splitted[10]);     } catch (exception e) {         logger.error(e.getmessage(), e);     }      string rowkeytext = "somtext";     immutablebyteswritable rowkey = new           immutablebyteswritable(bytes.tobytes(rowkeytext));      put put = new put(bytes.tobytes(rowkeytext));     put.add(cf,column, value);      context.write(rowkey, put);    }

hfileoutputformat or new version hfileoutputformat2 needs keyvalue final class. maybe putsortreducer not correctly applied converts put keyvalue instances.

in case not using mapreduce spark, creating keyvalue's directly instead of put's


Comments

Post a Comment

Popular posts from this blog

python - mat is not a numerical tuple : openCV error -

-
i have write down code showing error ma not getting it: please help: showing mat not numerical tuple: import cv import cv2 capture = cv2.videocapture("j.3gp") while(1): _, frame1 = capture.read() grayimage1 = cv2.cvtcolor(frame1, cv2.color_bgr2gray) _, frame2 = capture.read() grayimage2 = cv2.cvtcolor(frame2, cv2.color_bgr2gray) differenceimage = cv2.absdiff(grayimage1,grayimage2) thresholdimage = cv2.threshold(differenceimage,25,255,cv2.thresh_binary) cv2.imshow("difference image", differenceimage) cv2.imshow("threshold image", thresholdimage) cv2.imshow("image", frame1) k = cv2.waitkey(30) & 0xff error arising : ----------------------------------------------------------------------------------------- traceback (most recent call last): file "desk.py", line 15, in <module> cv2.imshow("threshold image", thresholdimage) typeerror: mat not numerical tuple ...
Read more

c# - MSAA finds controls UI Automation doesn't -

-
i'm working on automating windows application. i'm using teststack white framework. i've hit problem. program has 'window' object cannot see inside of. white shows no controls inside of it. inspect.exe shows no controls inside of either when running in ui automation mode. if switch inspect msaa see controls inside fine. there anyway me use msaa c# handle on these controls? if can identify msaa functions need, can use p/invoke call them c#. here's example article doing msaa: http://www.codeproject.com/articles/38906/ui-automation-using-microsoft-active-accessibility also, pinvoke.net can used identify iaccessible (msaa) functions: http://www.pinvoke.net/search.aspx?search=iaccessible&namespace=[all] here's answer along lines: msaa com-based? finally, alternative p/invoke, might able use tlbimp.exe create wrapper assembly oleacc.dll, , access msaa functions through it. i'm not sure if works msaa, it's worth try. as e...
Read more

wordpress - .htaccess: RewriteRule: bad flag delimiters -

-
pulling hair out on one. have following .htaccess file custom redirects in addition wordpress's default settings. when tested on local wamp server worked fine after moving production i'm not getting rewriterule: bad flag delimiters error in server log , after while site goes down internal server error any appreciated. in advance! <ifmodule mod_rewrite.c> rewriteengine on rewritebase / rewriterule ^rockford_weddings___welcome\.html$ /wedding/ [r=301,l] rewriterule ^blog/?$ /journal/ [r=301,l] rewriterule ^blog/(.*)$ /$1 [r=301,l] rewriterule ^portraitinvestment/?$ /portraitinvestment\.pdf [r=301,l] rewriterule ^weddinginvestment/?$ /weddinginvestment\.pdf [r=301,l] rewriterule ^holidaycards2014/?$ /holidaycards2014\.pdf [r=301,l] </ifmodule> # begin wordpress <ifmodule mod_rewrite.c> rewriteengine on rewritebase / rewriterule ^index\.php$ - [l] rewritecond %{request_filename} !-f rewritecond %{request_filename} !-d rewriterule . /index.php [l] </...
Read more