The problem with FPGAs is that they are a niche technology that is unable to exit its niche.
As someone who used FPGAs in a hardware design and regretted it later: there are two major problems with using an FPGA:
* it is too expensive for cost-sensitive devices,
* moving data onto and out of the chip is and always will be a problem.
Now, as to #1 above, you will hear FPGA enthusiast say that prices are just about to fall as more and more devices are produced. But I've heard such claims since at least 1998 and so far this simply hasn't happened.
As to #2, the number of MAC operations is irrelevant if you can't supply the chip with data. This is something even modern GPUs have trouble with, in spite of the monstrous bandwidth of the PCI slots they are placed in. Most algorithms will need to move data in and out, and use intermediate storage. Once you do all that, you end up with an expensive and complex design, that doesn't perform all that well anymore, and that is a pain to program and debug.
Also, if what you're doing is high-performance computing, then you have to compare the expenses with just getting a larger AWS instance or a cluster.
As a result, I tend to advise younger programmers to learn about FPGAs, but not become obsessed with them, as they are very rarely the right tool for the job. I'd even say they are almost never the right tool, except in very rare circumstances.
With modern FPGAs, you have the CPU on the same chip, which means you can use just one chip and then it's OK for cost-sensitive devices - and your problem of moving data across chips is gone as well.
I'm not an "FPGA enthusiast" myself - I work in chip design and Xilinx is a direct competitor. I think FPGAs have their share of problems preventing their adoption and I'll write a follow-up to elaborate. But the problems you mention are between solvable and solved.
(BTW it's true for GPUs just as well: just integrate them on the same chip, and forget about the PCI slots. Every high-end to mid-range cell phone has done it a long time ago.)
With modern FPGAs, you have the CPU on the same chip, which means you can use just one chip and then it's OK for cost-sensitive devices - and your problem of moving data across chips is gone as well.
The speed of the typical CPU on board an FPGA is ludicrously slow compared to the amount of data the FPGA itself can process.
BTW it's true for GPUs just as well: just integrate them on the same chip, and forget about the PCI slots. Every high-end to mid-range cell phone has done it a long time ago.
They pretty much have the same problem, exactly as OP stated. Nobody's using mobile phone GPUs for algorithmic acceleration. Hell enough of them have huge weak spots in doing texture upload/download themselves.
They're not all that much slower. Take a look at http://zedboard.org/. That's a Cortex A9 dual core proc attached to a hell of a lot of FPGA fabric. In general, I would expect that the speed of data "out" of an FPGA is going to be slower than the speed of data "in" to an FPGA, as well as being somewhat less data "out." I would expect to pull in a lot of ADC data and then filter and process and etc., only sending on what I absolutely want to keep.
I'm not saying that you can or should do this or that right now with a given piece of hardware, just what is and is not possible. The CPU on the FPGA's die could be fast, embedded GPUs could be better at algorithmic acceleration (here - wait for a few years, they have good stuff in their pipeline), OpenCL drivers could be easily available (of course portability would still be a real problem) and people could then program embedded GPUs more easily, etc. I've personally been developing accelerators integrated onto the same die as a CPU for a long time and it works nicely enough.
FPGAs are also stuck in a lot of niches. Economically, any high volume application where they could useful also means that putting an investment into more custom silicon is then feasible to get a better price margins - ASICs, DSPs, or I recently learned a bit about metal-gate arrays which seem to be mid-way between FPGAs and hard silicon designs.
They're stuck in a power niche because they're comparatively high power vs many embedded ICs, and if you're using a lot of power, as you said more desktop chips are economic to apply.
As someone who used FPGAs in a hardware design and regretted it later: there are two major problems with using an FPGA:
* it is too expensive for cost-sensitive devices,
* moving data onto and out of the chip is and always will be a problem.
Now, as to #1 above, you will hear FPGA enthusiast say that prices are just about to fall as more and more devices are produced. But I've heard such claims since at least 1998 and so far this simply hasn't happened.
As to #2, the number of MAC operations is irrelevant if you can't supply the chip with data. This is something even modern GPUs have trouble with, in spite of the monstrous bandwidth of the PCI slots they are placed in. Most algorithms will need to move data in and out, and use intermediate storage. Once you do all that, you end up with an expensive and complex design, that doesn't perform all that well anymore, and that is a pain to program and debug.
Also, if what you're doing is high-performance computing, then you have to compare the expenses with just getting a larger AWS instance or a cluster.
As a result, I tend to advise younger programmers to learn about FPGAs, but not become obsessed with them, as they are very rarely the right tool for the job. I'd even say they are almost never the right tool, except in very rare circumstances.