java - How to access Hadoop counters values via API? -

-

in hadoop can increment counter in map/reduce task, looks this:

... context.getcounter(mycountersenum.somecounter).increment(1); ...

than can find value in log.

how access them code after job completes?

what hadoop api read counter value?

counters represent global counters, defined either map-reduce framework or applications.

each counter can of enum type. can define counter enum in driver class

static enum updatecount{   cnt  }

and increment counter in map/reduce task

public class cntreducer extends reducer<intwritable, text, intwritable, text>{  public void reduce(intwritable key,iterable<text> values,context context)  {       //do       context.getcounter(updatecount.cnt).increment(1);  } }

and access them in driver class

public int run(string[] args) throws exception {  .  .  .  job.setinputformatclass(textinputformat.class);  job.setoutputformatclass(textoutputformat.class);  fileinputformat.setinputpaths(job,in );  fileoutputformat.setoutputpath(job, out);  job.waitforcompletion(true);  c = job.getcounters().findcounter(updatecount.cnt).getvalue();  //print "c"  } }

c gives counter value.

you can find example here


Comments

Post a Comment

Popular posts from this blog

python - mat is not a numerical tuple : openCV error -

-
i have write down code showing error ma not getting it: please help: showing mat not numerical tuple: import cv import cv2 capture = cv2.videocapture("j.3gp") while(1): _, frame1 = capture.read() grayimage1 = cv2.cvtcolor(frame1, cv2.color_bgr2gray) _, frame2 = capture.read() grayimage2 = cv2.cvtcolor(frame2, cv2.color_bgr2gray) differenceimage = cv2.absdiff(grayimage1,grayimage2) thresholdimage = cv2.threshold(differenceimage,25,255,cv2.thresh_binary) cv2.imshow("difference image", differenceimage) cv2.imshow("threshold image", thresholdimage) cv2.imshow("image", frame1) k = cv2.waitkey(30) & 0xff error arising : ----------------------------------------------------------------------------------------- traceback (most recent call last): file "desk.py", line 15, in <module> cv2.imshow("threshold image", thresholdimage) typeerror: mat not numerical tuple ...
Read more

c# - MSAA finds controls UI Automation doesn't -

-
i'm working on automating windows application. i'm using teststack white framework. i've hit problem. program has 'window' object cannot see inside of. white shows no controls inside of it. inspect.exe shows no controls inside of either when running in ui automation mode. if switch inspect msaa see controls inside fine. there anyway me use msaa c# handle on these controls? if can identify msaa functions need, can use p/invoke call them c#. here's example article doing msaa: http://www.codeproject.com/articles/38906/ui-automation-using-microsoft-active-accessibility also, pinvoke.net can used identify iaccessible (msaa) functions: http://www.pinvoke.net/search.aspx?search=iaccessible&namespace=[all] here's answer along lines: msaa com-based? finally, alternative p/invoke, might able use tlbimp.exe create wrapper assembly oleacc.dll, , access msaa functions through it. i'm not sure if works msaa, it's worth try. as e...
Read more

spring - @ResponseStatus and @ResponseBody not working with @ExceptionHandler -

-
i tried implement @controlleradvice globally handle general exceptions related web application. wanted send wrapped json response response status set accordingly. here code sample. @controlleradvice public class globalerrorhandler { @responsestatus(httpstatus.bad_request) @exceptionhandler(exception.class) @responsebody public response handleallerrors() { response wrapped = new response(httpservletresponse.sc_bad_request, "error"); wrapped.setmessage("could not retrieve data"); return wrapped; } } response pojo used wrap response. problem is, despite @responsestatus annotation, response has status code 500 (internal server error) default error page showing stack trace. @responsebody seems not work. documentation states send json response on @exceptionhandler method. don't understand i'm doing wrong in here. i'm using spring 3.2.8 release framework , gson used message conversion, if matters. a...
Read more