I did the same experiment as OP and ran into the same issues, but eventually realized that I was "doing serverless" wrong.
"Serverless" is not a replacement for cloud VMs/containers. Migrating your Rails/Express/Flask/.Net/whatever stack over to Lambda/API Gateway is not going to improve performance or costs.
You really have to architect your app from the ground-up for serverless by designing single-responsibility microservices that run in separate lambdas, building a heavy javascript front-end in your favorite framework (React/Ember/Amber/etc), and taking advantage of every service you can (Cognito, AppSync, S3, Cloudfront, API Gateway, etc) to eliminate the need for a web framework.
I have been experimenting with this approach lately and have been having some success with it, deploying relatively complex, reliable, scalable web services that I can support as a one-man show.
> You really have to architect your app from the ground-up for serverless by designing single-responsibility microservices that run in separate lambdas, building a heavy javascript front-end in your favorite framework (React/Ember/Amber/etc), and taking advantage of every service you can (Cognito, AppSync, S3, Cloudfront, API Gateway, etc) to eliminate the need for a web framework.
At least I don't have to learn that complex "systems admin" stuff.
Exactly, it's like saying to someone running a restaurant that buying their bottled water from a convenient store is more expensive than buying it in bulk from Costco.
It's entirely missing the point. At the end of the day, you have to look at your specific usage pattern and pick the best option for you. Obviously, as with any other technology, anyone who forces a specific option in every possible situation is most likely wrong.
To eliminate the need of a web framework? I don’t understand the rationale, if I can get all what’s mentioned done with a good web framework, I will be more than happy to do that.
With your own server and Web framework, you do all the work in provisioning the machine, configuring services, installing dependencies, building deployment and integration pipelines and, worst of all, maintaining all that when updates are released / something breaks. It is also harder to scale.
A serverless solution that eliminates the Web framework (and thus the stack in which is being run) does most of that for you, at the expense of extra cost or infrastructure deployment complexity, but once it is done, scaling and maintenance are easier.
Firebase now makes most of these painless. They've done a really good job. If your starting from the grounds up and can stomach using a google product Firebase is the easiest to work with by far.
First and foremost it is client side SDK's (web, mobile) for their database products, their newest being Firestore that provides better query capabilities compared to their original Firebase Realtime database (while still offering real-time capabilities).
Along with that is Firebase Authentication, which manages user accounts and authentication.
The real magic comes in with Cloud Functions (their version of Lambda) which allows for hooks in to all sorts of events occurring from usage of these database and authentication products (and other cloud services).
Hook into database writes, updates, deletes, user creation, Google Cloud's pub-sub events and many more. They also offer static website hosting as well as hooking website serving into cloud functions (for server side code execution).
In the context of a website, all of these work together to allow for usage of the JAMstack[0] architecture which decreases your infrastructure resources you need to manage and cost.
Firebase does a lot of stuff. Originally it was a small company that focused on providing a real-time JSON-like backend store for web apps. But then they got bought by Google and seem to have evolved into Google's answer to a lot of AWS services, ie hosting, real-time DB, serverless, and probably more I'm not aware of.
I've often looked at and played with Firebase since it does so much of what I need to back a React Native-based app for simple mobile games and utilities. I always end up talking myself out of it due to Google's history of pulling the plug on (what seem to an outsider to be) perfectly good, stable products that wouldn't do any harm to keep around indefinitely.
As a hobbyist, I wouldn't have the time or motivation to completely rewrite a project if that happened, which would be necessary since a Firebase app (like a heavily AWS-integrated serverless app) is not just technologically but architecturally tied to that environment.
It seems like a pretty reasonable concern to me. We can presume the odds are pretty low, yeah. But the consequences are very high - you would need to redesign and possibly rewrite in another language your entire application. People complained up a storm about Reader dying, but it was a 5min process to export your subscriptions and import into another web reader that had basically the same feature set. Conventional Linux hosting, or even Docker, could be pretty easily re-hosted as-is on any of hundreds of other places.
I usually go with a Django or RoR monolith, something that I know I can run on a cheap DO droplet or similar, and while there's a fixed monthly cost it's reasonable and I can scale up with ease (albeit manually). If I were explicitly needing the realtime DB aspect I'd probably look to Phoenix with the same hardware approach.
It would be nice if there was a dead-simple Firebase-like tool you could self-host. ie just a single instance that you could point all your toy apps at to give them a little real-time persistence.
"Serverless" is not a replacement for cloud VMs/containers. Migrating your Rails/Express/Flask/.Net/whatever stack over to Lambda/API Gateway is not going to improve performance or costs.
You really have to architect your app from the ground-up for serverless by designing single-responsibility microservices that run in separate lambdas, building a heavy javascript front-end in your favorite framework (React/Ember/Amber/etc), and taking advantage of every service you can (Cognito, AppSync, S3, Cloudfront, API Gateway, etc) to eliminate the need for a web framework.
I have been experimenting with this approach lately and have been having some success with it, deploying relatively complex, reliable, scalable web services that I can support as a one-man show.