I don't think so. I'm aware of a tolerance (+/- 0.25V I think), as well as a requirement for a host to be able to source 100-500mA as negotiated. To my understanding that negotiation is seldom implemented.Does the USB specification deal with this? I mean, if every device the OP might charge with his device will be smart and disconnect itself when it sees low voltage on the USB host, maybe the OP doesn't need a low-voltage shutoff at all.
From what I've seen of USB power interfaces as implemented on DIY-type projects (eg a portable power pack), the 5V is supplied from batteries using a DC-DC boost converter. So I guess that the 5V would be maintained for an open-circuit USB output, but that would sag pretty quickly as current is drawn when the host's batteries run low. In this case the host would be responsible for a reliable shut-down to protect it's own batteries from damaging deep discharge, not necessarily for the benefit of the connected client device.
This self-preserving issue aside (it doesn't apply to solar batteries), I gotta assume that undervoltage supply conditions happen all the time in the USB world (like, USB wall warts rated at 200 mA never get overloaded?...). I can imagine some client device sucking in whatever little power is offered by a host via DC-DC boost conversion. If I had to recharge my batteries from such a host, that's what I would do. From that point of view, I would leave it up to the client device how to react to undervoltage conditions.