Just going to plug a book here that was an awesome read in my undergrad on that topic, and the history of quant finance over the 20th / early 21st centuries: https://byjustinfox.com/myth-of-the-rational-market/
As an Australian, your second point is hard to vindicate. The total death toll in Australia is 106 as of one hour ago as reported by the state-funded ABC[0].
While it's true that just one of those was a person under the age of fifty[1], and unfortunately I live in the _one_ state that's experiencing a second wave via community transmission and sadly expect these numbers to grow, this isn't comparable with the numbers seen coming out of Sweden.
In fact, it's an order of magnitude lower, for a country with more than double Sweden's population, so to suggest that at-risk groups are experiencing a similar level of risk is baffling to me.
62/106 deaths from COVID in Australia have been in nursing homes[1][2].
The point isn't that Australia hasn't done well (it has!). The point is that even in the best possible circumstances it still proved impossible to keep nursing home patients safe.
In Sweden the plan was to keep nursing homes isolated too, and it didn't work there either.[3]
Sorry about missing you were down under, I only checked after hitting submit. And thanks for clarifying, I had no idea Sweden moved to isolate nursing homes.
That said, I'm still hesitant to equate the risk level to the elderly from community-wide lockdown measures with isolation policies restricted to nursing homes. I don't have any data to support it, but I can't help but assume that lower community transmission overall correlates with fewer transmission vectors into nursing homes. Supposedly this would turn up in the proportion of those in nursing homes that's suffered coronavirus. In principle I agree with you; the hotel scandals in Vic bear that out, ironically, and sadly. But the difference in magnitude has to count for something, as bad as any toll is.
I can tell a story. I used to work for a HVAC installation company, pretty small in terms of staff but we subcontracted a lot. Initially brought on as a mechanical engineering intern, but moved on to sales engineering when I found an interesting statistical relationship.
A large factor in quotes to clients was the underlying cost of air conditioning equipment in our niche, and often a game of sales intel was played between suppliers and competing contractors (like us) for a given job site. Favorites were picked, and we could get royally screwed in a quote, losing the sale to the end-customer.
Fortunately, we had years of purchasing information. It turns out that as varied as air conditioners are across brands and technical dimensions, when you have years of accounts' line items and unused quotes, you don't get a dimensionality issue. Since we operated in a clear-cut niche, this was especially true. We could forecast, within a margin of error of two per cent, exactly what any of our suppliers would quote us (or our competitors!) for a job long before they could turn it around. Huge strategic advantage.
This was the watershed moment for me when I realized even basic multiple linear regression was a scarily powerful tool when used correctly.
And incredibly boring. The usual estimate is that data science is 80% data wrangling: finding, collecting, and cleaning up data. The term "data scientist" replaced "data miner", because miners are looking for gold. Scientists are obsessed with finding out the nature of reality, gold or mud. They will do seriously boring stuff to set things up so that reality is revealed.
If the data cleaning is follows standard patterns, you should already have scripts to offload that kind of work to. If not, then there some incredibly interesting decisions hidden underneath. Like in text: Should character casing be preserved ? What should be the unit of representation (word/character) ? How should data be filtered: Quality vs quantity trade-off ?
All of those are non-trivial questions which involve a lot of thought to reason through. You are correct that the modelling is only a small part of DS's day to day job.
But, the rest of it is boring in the same way that coding is boring. It is doesn't involve some grand epiphanies or discoveries, but there is joy similar to the daily grind of "code -> get bug/ violate constraints -> follow trace/problem -> figure a sensible solution" that a lot of software engineers love.
Consider adding a logarithmic y-axis to each graph, but otherwise, nice presentation!
Edit: also, perhaps y-axes of the form "Thousands of X", then increments can show "200, 400, 600..." and so on. Simpler to digest when lots of zeroes come into play, at least for me.
Great point, and I think this is a next step for me. For what it's worth, this was to teach myself React and Redux and took a couple of nights of work. I was curious about ways to keep the canvas element fundamentally decoupled from the algorithm itself, which allows a couple of benefits out of the box. For example, varying the randomness or changing the color map mid-calculation, I kind of got 'for free', thanks to React re-rendering the canvas every time the algorithm is stepped through or the controls are changed.
It's also worth noting that the algorithm's speed here is tied to (I believe?) your monitor's refresh rate, since it's using requestAnimationFrame to loop without blocking the main thread entirely. I'd briefly considered a "complete" button which would run the whole thing in one go and render once, and that would enable reviewing for performance optimisation without hanging the page. But... I got too excited and wanted to share it first.
For the smaller sizes it's fast enough, but for larger sizes I'd recommend showing completed chunks of steps at a time, instead of each individual diamond/square step at a time.
And the algorithm is "embarrassingly parallel" so you can run it really efficiently on the GPU if you want to make some huge maps.
Effectively, yes. It's a multiplier for the random displacement that's calculated for a point in the grid as part of the algorithm, which is added to the mean of the adjacent (diamond- or square-wise) points.
Agreed. There's not much that will bake freshly learned theory into your brain more than realising just how far it could have taken you in hindsight. Over time, this starts to develop into foresight.
