I find it interesting that you (implicitly) like Data Munging better than Clever Algorithms. I agree that the former is orders of magnitude more common. I haven't encountered a Clever Algorithms problem very often at all on commercial projects. My current project is an outlier insofar as we've been doing algorithmic work for months. It's fun, in the way any good hacker would expect such a challenge to be, but it's also stressful to work on something for a long time and not know for sure whether you'll end up with a solution. (Especially when the context is a startup and not a research project.) It's much easier when you know for sure you can get there, you just have to figure out how. Thankfully, the Algorithms category gets more and more fun as you knock off significant chunks of the problem. Once you see the light at the end of the tunnel, you're home free. Unless, of course, you got it wrong. :)
Edit: by the way, do you actually use APL/J/K in the way you mentioned? I really like that paradigm. I think it deserves a place alongside OO and the others as a fundamentally different approach to programming. However, I haven't used it nearly enough.
Edit 2: I would distinguish two different kinds of Data Munging. One is Data Transformation, where you have data in some original context and it needs to go through a series of transformations or passes until you've extracted or computed what's suitable for your problem. That is way fun. The other type, though, sucks rocks. I call it Meat Grinding. That's when you have the exact data you need in a form that you can't use because of some purely technical limitation. A common example is you have data in a database or a DTO when what your code need is a domain object, so you have to copy the properties over like this:
This isn't so much a programming problem as an integration one. (You sort of touch on it at the end.) It's also one of the worst things about programming in less powerful languages where you can't easily write a program to do the gruntwork for you.
Edit 3: I'd also argue that your "API spelunking" nightmare is at its nastiest in OO systems. In poorly designed pre-OO APIs (like Win32), it might take a long time to figure out how to call a function or what to do next; there might have been some ugly one-off struct you had to populate, etc.; but at least you didn't have to figure out how the hell to make a Bar out of a Foo so you can pass it as the third argument to Baz, when neither Foo nor Bar has anything to do with your problem. But with poorly designed OO APIs, these absurd hurdles seem endless. I used to compensate for this by studying such code archaeologically (if I couldn't avoid it altogether), in order to learn how the absurdities had arisen over time. This was a way to escape Dostoevsky's definition of the easiest way to drive a man insane: make him move a pile of dirt from one end of a prison yard to another and then move it back.
I understand your use of the pronoun "you", but it's not me. I work almost exclusively on algorithmic stuff, and when I do work on data munging it's the direct application of algorithms just developed. So I'm lucky.
Edit: by the way, do you actually use APL/J/K in the way you mentioned? I really like that paradigm. I think it deserves a place alongside OO and the others as a fundamentally different approach to programming. However, I haven't used it nearly enough.
Edit 2: I would distinguish two different kinds of Data Munging. One is Data Transformation, where you have data in some original context and it needs to go through a series of transformations or passes until you've extracted or computed what's suitable for your problem. That is way fun. The other type, though, sucks rocks. I call it Meat Grinding. That's when you have the exact data you need in a form that you can't use because of some purely technical limitation. A common example is you have data in a database or a DTO when what your code need is a domain object, so you have to copy the properties over like this:
This isn't so much a programming problem as an integration one. (You sort of touch on it at the end.) It's also one of the worst things about programming in less powerful languages where you can't easily write a program to do the gruntwork for you.Edit 3: I'd also argue that your "API spelunking" nightmare is at its nastiest in OO systems. In poorly designed pre-OO APIs (like Win32), it might take a long time to figure out how to call a function or what to do next; there might have been some ugly one-off struct you had to populate, etc.; but at least you didn't have to figure out how the hell to make a Bar out of a Foo so you can pass it as the third argument to Baz, when neither Foo nor Bar has anything to do with your problem. But with poorly designed OO APIs, these absurd hurdles seem endless. I used to compensate for this by studying such code archaeologically (if I couldn't avoid it altogether), in order to learn how the absurdities had arisen over time. This was a way to escape Dostoevsky's definition of the easiest way to drive a man insane: make him move a pile of dirt from one end of a prison yard to another and then move it back.