Hey Kayla,
some time ago I created a similar app which is detecting hand poses for a rock-paper-scissors game and I choose ml5.js. In my opinion, they provide a very convenient way of abstracting the complex parts of machine learning away from you and with p5.js combined you have a powerful setup to handle your requirements.
Here are some links to start, the first one from the ml5.js website with a hand pose example, the second one is my own hand pose project with vue.js, which you could take as a basic setup and would just need to scan in your signs, and you're good to go.
https://learn.ml5js.org/#/reference/handpose?id=handpose
https://github.com/lmarschall/rockpaperscissors/blob/main/app/src/views/About.vue
Very interesting project, by the way, hit me up if you have any questions.