If you are going to build from scratch use Laravel, because it is a little easier to learn than Symfony. They have a bunch of great videos to help you along the way. If you know Drupal 8 already, that is built on Symfony and you can harness the backend, but it is going to have a steeper learning curve. On the plus side you can take advantage of all its features. I wouldn't recommend building without a type of framework. Thousands of man hours have gone into those things for a reason. I started learning Symfony w/o Drupal but lost interest once I found out how much easier Laravel was. They both have their advantages and disadvantages, laravel actually uses part of symfony in its code. I like the blade template system better than twig is a big factor in deciding as well. They are both very similar, but blade is closer to native PHP which makes it a little easier to learn.
I use Laravel because it has integrated unit testing that making TDD a breeze. Having a View (Blade engine) making me easier to work without too many efforts in front-end.
I do recommend going into the root of programming once getting stable on any framework. Go beyond Symfony, go beyond PHP, go into the roots to the mother of programming; c++, c, smalltalk, erlang OTP. Understand the fundamental principle of abstraction.
A framework is just a framework, it helps in getting feedback quickly; like practicing dancing in front of a mirror. Getting fundamentals right is the one true key in doing it right. Programming is not hard, but abstract-programming is extremely hard.