This solar panel power bank was supposedly waterproof but accidentally left out in a storm and the USB ports and circuit board were toast. It was 20,000 mah. I took the casing and circuit board to the trash can and just have the panels and batteries.
I am considering doing some long hikes so I was thinking of attaching the panels to my backpack, which is basically already an existing product. Or even just one single lightweight panel.

I don't need the whole 20,000 which weighs a lot so was thinking of just wiring the panels to one of the 2 battery packs.

I am really a beginner but my understanding is the battery would charge. Next question is how to charge a phone from that.

Can I just take a smartphone charger cut off and wire that to the battery? Or is this where I would need some kind of voltage controller.

Side question could I just wire straight from the panels to the smartphone charger?

 

You need control circuits in between the panels and a battery and in between the battery and a phone. There are dedicated battery charging IC chips you can buy to use a solar panel to charge a battery. Some of these even include output control as well, ie. both controllers in a single package. The output you need is probably 5V USB power, and that's very common.

But I think the bottom line is that, especially for a beginner, the project won't be worth your time unless you enjoy the project for itself. If your goal is just something that works, search Amazon and eBay for the appropriate device already built. Take a look at this, for instance:
https://www.amazon.com/Controller-Intelligent-Regulator-Paremeter-Adjustable/dp/B08L8TBCK6
I'm not suggesting this particular device is a perfect solution for you but it's a good example of the type of thing you can find. Lots of features, not expensive.

 

It's probably not worth bothering with the solar panels. They're not going to collect much energy under ideal conditions, and even less when hung on a backpack. So all you really need is the power bank stuff, a board that will safely charge the cell from USB, and boost it to 5V for charging or powering stuff. If you search ebay for power bank module, you'll find a bunch. (There might be some that include solar charging; if that's an option, keep the solar panels.) If your lithium cell doesn't have a protection board (disconnects the cell in case of short circuit, over-charging, or over-discharging) attached, it would be a good idea to add one of those as well. Search for 1S BMS.