There is no reason why, in principle, this is not possible, Whether it is practical of not is a different question.
In commercial optical flow sensors targeting small UAVs, the output is just a simple XY displacement. This makes the flight controllers job of correlating the optical flow with altitude and position information much easier.
While your approach seems doable, it will require replacing the dedicated conversion hardware in the optical flow sensor with flight controller CPU—and I am not sure that is practical.
The actual sensor is only part of the system in the sort of optical flow sensor used in a sUASs, and if it was practical to use software to replace the extra hardware that would seem to be a cost savings manufacturers would jump on.
Some mouse sensors offer the three critical signals separately—ΔX, ΔY, and image quality—but while that reduces computational cost over the type that expose only raw pixel data it will still requite extensive integration of the IMU data to account for apparent motion caused by aircraft roll, pitch, and yaw—something a mouse doesn't deal with.
One possible strategy would be to front-end the sensor with a small FPGA to do the heavy lifting for the MCU in the FC. The result would be nicely cooked data for the FC to consume and a greatly reduced CPU utilization which I surmise will be the biggest challenge.
N.B.: The foregoing is speculative. I have not tried to do this myself, and I could be wrong, so free advice is like all ofter free things—you tend to get what you pay for.