We have microservices in production with both technologies (hosting simple CRUDs, more complex async processes, GraphQL endpoints, etc). As a matter of preference, for greenfield projects, we tend to go full serverless (lambda, dynamo, aurora...), because is not only cheaper in general, but way easier to continuously integrate with bitbucket pipelines, github actions, etc. The drawback is still the cold starts, despite the last improvements in VPC cold starts; it's way less noticeable in non statically typed languages though (Node.js, Python...). You will have to keep the lambdas warmed with Cloudwatch events if you can't wait for a few seconds after the first invocation, or use the new feature Provisioned Concurrency (which we haven't), that kind of defeat the purpose of being serverless.
As to how to host different methods of an API in lambdas, there are different approaches: we have for example stacks of cloud formation where every lambda host unique functionalities (one for detecting faces in images, another for cropping, another for comparing, etc), and also other stacks where the entire GraphQL API is hosted with Fastify in just one lambda. It really depends on your use-case.
If you host your API in ECS with an ALB, you won't have cold starts, it will automatically handle blue/green deployments with no disruption in the service while you deploy, but your bill will increase, and the infrastructure will be more difficult to automate in something like a template.yaml.
Give it a try with aws sam (Serverless Application Model), with a few commands, you can have a sample API running in a couple of minutes.