Load balancing lithium ion batteries can be challenging. I would definitely use a charge controller, for safety. It might work fine just wiring then together at first, but unless they were EXACTLY the same cell voltage the weakest one is going to degrade over time, and you can’t really tell when it’s tucked away in an enclosure. Lithium fires are no joke.
The terminology they use for charge controllers is a number of cells and a letter for wiring configuration. So in your case you want a “4S” charge controller, for “four cells in series”. Here is one that should work for your purposes from SMAKN, who I’ve ordered power supplies from before.