> my rule of thumb is that anything that can be done inside the database should be done inside the database

For non-distributable (or at least non-multi-master) DBs, you are often using your most expensive and least distributable resource to do your cheapest and most distributable work. My rule of thumb is that unless you need atomicity or the performance (as profiled) makes a real difference, do it outside the DB. Also has the side effect of avoiding system lock-in (at the expense of language lock-in which I think is ok).

Granted, it's subjective, but I've had the most trouble scaling DBs than other systems, and it's usually CPU/mem of business logic pushing the boundaries.

I don't often find myself in a situation where I don't need need atomicity for the work I do. So I'm speaking from a point of view that reflects that.

Even if I didn't need it for a project, I would probably start there because, you know, that's the hammer I have, so SQL is the nail I'm going to start with.

I haven't had problems scaling DBs though. DB throughput has never caused a problem for me. What has caused problems is business logic put in what I consider the wrong places.

Business logic belongs in the database structure wherever possible because that's the single source of truth for your data. That's where all of your controls for data integrity belong.

For any app that has longevity, you are going to have multiple methods of accessing the data. There will be APIs and crappy junk that wants your data. You have to account for this. There will be users logging directly into your database. Sometimes it will be your boss.

Start with an ACID approach to your CRUD app, and you probably won't go wrong. At least not soon. And you have options for scaling when you need to. You can scale vertically or horizontally or both.

> For non-distributable (or at least non-multi-master) DBs, you are often using your most expensive and least distributable resource to do your cheapest and most distributable work.

I see your logic, but I think this is misleading, and often the reason why people do inefficient things.

The pattern I see regularly is if the database does more thinking, its planner will diminish the amount of actual work; it will touch fewer rows from disk, and less data will result sent over the network and in fewer round trips. Overall for any extensive report you're likely to be paying significant cost to distribute the work, often more than actually doing the work.

Of course, as you say it's subjective. Scaling databases becomes necessary at a point, but it may be because of developers not embracing them; I think this is why NoSQL has gained in popularity.

Exactly, you really need ACID only for important stuff, other data you can cache on workers where you run most of the logic

If you're doing reports you could run them against a read-only slave.

> It's really manageable. It requires that your dev team really knows SQL

I'd question the credentials of any developer who doesn't know SQL. It's an essential skill across the entire IT-indutry. Not knowing it means you're obviously not serious about your job.

And here you're talking like programmers not knowing it is a normal thing... Are we working in the same industry?

Feel free to question my credentials, then. I'm 44, I've been a dev all my life, and I've never worked with SQL. I've worked with distributed in memory data grids, strange custom databases specific to trading systems, and many other data storage systems, but I've literally never written a single SQL query that has gone to test or production. I've also written tons of code that hasn't stored data, including my current project (an IDE).

There's a lot more to life than CRUD apps.

The chain of comments leading up to yours is interesting. One person said that if you've got a relational database you should push lots of work down to it, which they say requires lots of SQL knowledge, and they think having that knowledge is good. Then another person replied and said you're not a real programmer if you don't know SQL. Then you pointed out that it's entirely possible to be a developer and never touch relational databases.

I'd like to support your comment, and argue for a more restrictive claim. Namely, a developer is more valuable the better they know their tools. If they're working with a relational database, then it's good for them to learn more about how to use it. However, if you're running a team that is building some software system, you ought to make technology decisions which make sense given the skills of your team members; so, if the team's SQL knowledge is weak and there aren't compelling performance reasons to push lots of work into the database, it's probably not the right choice.

I appreciate your tone and like your comment (read: upvoted). But I want to respectfully disagree with it.

I don't agree with the part of your comment that says, essentially, work with the tools your team already has.

There are some cases where you have to choose the technology that makes the most sense and your team has to get with the program. If you're storing data in any way, you have what I think of as an obligation to choose the best technology to store it.

I know and have worked with people who would write everything to a csv file because they know how to do that and prefer to do that. And toss it on a server somewhere with no redundancy or backups of any kind.

We all come from different places and have different jobs. But some times you have to suck it up and learn a new thing to be effective in this industry.

Sometimes a CSV file actually is the right answer. Sometimes a database of some flavor is the right answer.

The point I was trying to make and didn't do such a good job of was that if you're dealing with a web app, a database is probably the right answer, and if you're using a database, you should maximize what it can do for you.

I should have made my comment more explicit. I'm talking about web development.

There are a ton of jobs which my comments don't apply to. And yours sounds like one of them.

There are are entire areas of my work that also have nothing to do with collecting or storing data, and I recognize that SQL has no place in that work.

I was speaking to the bulk of my work and assuming (perhaps wrongly) that we were talking about similar things.

My bad. And you are totally correct about life being more than CRUD apps.

To be clear, I was replying to josteink who said: It's an essential skill across the entire IT-indutry. Not knowing it means you're obviously not serious about your job. I thought your comment was totally reasonable :-)

Sometimes when you question you'll find that the person really has the chops :-)

I'm sure you'll agree however that for every one of you, there's ton of developers who are less programmers and more assembly line workers gluing framework pieces together :-)

There is a large span between "knows SQL" as in "can write and read basic SQL query" and "really knows SQL". And I bet there are tons of developers who never have had the need to touch SQL or even any RDBMS at all. It's an essential skill in parts of the industry, it has no practical relevance in others.

Upwork.com is a great way to fill that gap: tons of great SQL guys, very reasonable cost.

How much SQL does one have to know before they are "serious about their jobs," in your view?