Please consider Scalyr: disclosure I work there, but objectively I will be surprised if someone beats us on price, focused log analytics - try and judge for yourself.
Does the speed matter? How exactly?
I am genuinely curious: at Scalyr we _can_ be very fast but it is a balance with the cost that we want to pass on as price savings. Same with self-hosted Elastic: one can fine-tune it to be fast but minding the cost constraints gets it slower. WDYT?
A story of Netflix adopting StackStorm for autoremediation is a nice illustration of this approach. They began to build a tool, learned enough to be dangerous, discovered StackStorm and used it, experimented with it like with a breadboard, until finally figured out exactly what works - and reimplemented it. All these time learning the system was working and delivering vale.
The dichotomy is real and a reflection of Dev vs Ops dichotomy. DevOps made Dev and Ops collaborate but didn't blend the roles & skills. Ops appreciate logs but require consistent metrics to identify and root-cause the problem. Dev appreciate metrics but require logs to debug and fix the problem. Opinions on what is more important are informed by role and experience; the author makes it clear that as a team, we need both.
> For many of the services I run, I haven't looked at logs in months, because the metrics tell the story. If service is degraded, I can usually correlate it to another downed service, a network failure, or a recent change. No logs needed.
Good point, echoing Brendan Gregg, the author of "USE" method commented:
> The USE Method is based on three metric types and a strategy for approaching a complex system. I find it solves about 80% of server issues with 5% of the effort,” (http://www.brendangregg.com/usemethod.html)
Solving 80% of issues with 5% of effort is commendable; the rest 20% goes to developers where the other 95% of effort is spent debugging and fixing the problem, primarily by reasoning about the logs.
So:
- "which of metrics or logs is more important?" is a relative and moot
- "can metrics be extracted from logs?" - yes; "is it practical?" - it depends: likely NO for DIY. The fact that ELK is not making it particularly well doesn't mean that other products can't / don't do it.
-
On top of "this sucks": such "fully automated" response is a violation of GDPR compliance that requires "right to obtain human intervention on the part of the controller".
Stunning that Google cloud can get away with this.
Nurtch DOES look like an awesome idea: jupyter notebooks proved to be a greatest interactive tool as well as for sharing the work. Good job with the site, too, best of luck with it.
If you’re serious about automating operations and wiring with slack/Chataops, check out StackStorm (disclosure I am one of the creators). Open source Apache 2.0,?installable, no mines. It’s much heavier than Nurtch or FireHydrant, as it comes with IfThisThanThat logic and workflows to string individual actions in runbooks, chat ops out-of box, but different strokes for different folks, we all runnops differently, pick your pill. https://GitHub.com/StackStorm/st2
Thanks & good work with StackStorm :) I'm curious to know your take on Rundeck as well?
For others: StackStorm/Rundeck offers IFTTT for operations (amongst other things). Alert X -> Run script/workflow Y. It's an ideal approach to operations as it takes human out of the loop. Less on-call alerts, quick recovery. Sort of like a self healing system. Highly recommend.
Some companies can't/don't want to invest in full automation workflows. Nurtch is useful in this case as it allows partially automating runbooks by interleaving instructions and code. Downside is that human is still involved, engineer still gets paged :( but on the plus side, resolution could take 10 minutes instead of 2 hours & less upfront investment.
Rundeck is a good system, around for quite some time. I met many people who are using it in prod, some migrating to StackStorm when they need IFTTT, or those who prefer not to operate Java, some staying as Rundeck has more UI for one-off commands.
If the comp, stocks, big $ is your primary motivation, look somewhere else.
We go to the startup for a thrill. For doing something you strongly believe. For creating something that didn't exist before. If you think you can do it in a big org, too, you are right, now go and try, get a job at Cisco or AT&T. Innovating there is like driving Kia Rio vs Porsche: it's a car, too. It goes thought the motion. But the thrill is not there.
Many people are happy in big companies. They won't be happy at startups. They're no worse because of this. Different stokes for different folks. There is no objective pro's and con's for startup vs big corp. There are properties, that match personal traits. The key is to know yourself and what are yours. Trouble is, we don't always know what we want, worse yet it changes over time with experience. And once we finally learn it, it's time to retire. Life is fun.
Now, you don't want to be taken advantage, right, so all written here about transparency, cap table, etc are the right things. Do check. Don't take below-market salary for the mirages of future. If you don't do these basics - you failed intelligence test and shouldn't be hired.
There aren't nearly as many people out there who want the thrill of working for a startup more than they want higher comp AND can afford to miss out on big-co comp as there are startups wanting to hire them.
If startups want to be able to fill job openings (the question posed in this Ask HN), they have to be able to offer something besides thrill.
Second paragraph is great. I haven't worked at a startup, but my experience in a big company is that you can definitely innovate, change things, and make things better.
But I wouldn't call it thrilling, I'd be interested to see how many startup employees found it thrilling.
https://www.scalyr.com