logstash - Is it possible to have Centralised Logging for ElasticBeanstalk Docker apps? -

-

we have custom docker web app running in elastic beanstalk docker container environment. have application logs available viewing outside. without downloading through instances or aws console.

so far neither of solutions been acceptable. maybe achieved centralised logging elastic benastalk dockerized apps?

solution 1: aws console log download

not acceptable - requires download logs, extract every time. non real-time.

solution 2: s3 + elasticsearch + fluentd

fluentd not have plugin retrieve logs s3 there's excellent s3 plugin, it's log output s3. not input logs s3.

solution 3: s3 + elasticsearch + logstash

cons: can pull logs entire bucket or nothing.

the problem lies elastic beanstalk s3 log storage structure. cannot specify file name pattern. it's either logs or nothing. elasticbeanstalk saves logs on s3 in path containing random instance , environment ids:

s3.bucket/resources/environments/logs/publish/e-<random environment id>/i-<random instance id>/my.log@

logstash s3 plugin can pointed resources/environments/logs/publish/. when try point environments/logs/publish/*/my.log not work. means can not pull particular log , tag/type able find in elasticsearch. since aws saves logs environments , instances in same folder structure, cannot chose instance.

solution 4: aws cloudwatch console log viewer

it possible forward custom logs cloudwatch console. achieve that, put configuration files in .ebextensions path of app bundle: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/awshowto.cloudwatchlogs.html

there's file called cwl-webrequest-metrics.config allows specify log files along alerts, etc. great!? except configuration file format neither yaml,xml or json, , it's not documented. there absolutely 0 mentions of file, it's format either on aws documentation website or anywhere on net. , 1 log file appear in cloudwatch not adding configuration line. possible way working seem trial , error. great!? except every attempt need re-deploy environment.

there's 1 reference how make work custom log: http://qiita.com/kozayupapa/items/2bb7a6b1f17f4e799a22 have no idea how person reverse engineered file format.

cons:

  • cloudwatch not seem able split logs columns when displaying, can't filter priority, etc.
  • aws console log viewer not have auto-refresh follow logs.
  • nightmare undocumented configuration file format, no way of testing. trial , error requires re-deploying whole instance.

perhaps aws lambda function applicable?

write javascript dumps notifications, see can those.

after object written, rename within same bucket?

or notify own log-management service creation of new object?

lots of possibilities there...


Comments

Post a Comment

Popular posts from this blog

python - mat is not a numerical tuple : openCV error -

-
i have write down code showing error ma not getting it: please help: showing mat not numerical tuple: import cv import cv2 capture = cv2.videocapture("j.3gp") while(1): _, frame1 = capture.read() grayimage1 = cv2.cvtcolor(frame1, cv2.color_bgr2gray) _, frame2 = capture.read() grayimage2 = cv2.cvtcolor(frame2, cv2.color_bgr2gray) differenceimage = cv2.absdiff(grayimage1,grayimage2) thresholdimage = cv2.threshold(differenceimage,25,255,cv2.thresh_binary) cv2.imshow("difference image", differenceimage) cv2.imshow("threshold image", thresholdimage) cv2.imshow("image", frame1) k = cv2.waitkey(30) & 0xff error arising : ----------------------------------------------------------------------------------------- traceback (most recent call last): file "desk.py", line 15, in <module> cv2.imshow("threshold image", thresholdimage) typeerror: mat not numerical tuple
Read more

c# - MSAA finds controls UI Automation doesn't -

-
i'm working on automating windows application. i'm using teststack white framework. i've hit problem. program has 'window' object cannot see inside of. white shows no controls inside of it. inspect.exe shows no controls inside of either when running in ui automation mode. if switch inspect msaa see controls inside fine. there anyway me use msaa c# handle on these controls? if can identify msaa functions need, can use p/invoke call them c#. here's example article doing msaa: http://www.codeproject.com/articles/38906/ui-automation-using-microsoft-active-accessibility also, pinvoke.net can used identify iaccessible (msaa) functions: http://www.pinvoke.net/search.aspx?search=iaccessible&namespace=[all] here's answer along lines: msaa com-based? finally, alternative p/invoke, might able use tlbimp.exe create wrapper assembly oleacc.dll, , access msaa functions through it. i'm not sure if works msaa, it's worth try. as e
Read more

wordpress - .htaccess: RewriteRule: bad flag delimiters -

-
pulling hair out on one. have following .htaccess file custom redirects in addition wordpress's default settings. when tested on local wamp server worked fine after moving production i'm not getting rewriterule: bad flag delimiters error in server log , after while site goes down internal server error any appreciated. in advance! <ifmodule mod_rewrite.c> rewriteengine on rewritebase / rewriterule ^rockford_weddings___welcome\.html$ /wedding/ [r=301,l] rewriterule ^blog/?$ /journal/ [r=301,l] rewriterule ^blog/(.*)$ /$1 [r=301,l] rewriterule ^portraitinvestment/?$ /portraitinvestment\.pdf [r=301,l] rewriterule ^weddinginvestment/?$ /weddinginvestment\.pdf [r=301,l] rewriterule ^holidaycards2014/?$ /holidaycards2014\.pdf [r=301,l] </ifmodule> # begin wordpress <ifmodule mod_rewrite.c> rewriteengine on rewritebase / rewriterule ^index\.php$ - [l] rewritecond %{request_filename} !-f rewritecond %{request_filename} !-d rewriterule . /index.php [l] </
Read more