Set up your mesh equations taking care to compute the voltage across each inductor according to the dot conventions. Just remember that positive changing current entering the dot of one inductor maps to positive voltage at the dot of the other inductor.
Could someone create a simulation of the circuit and use a simulated watt-meter? Using trial and error the value of R could be determined. Then I would at least know what the correct final answer is.
Well, what did you get for the final resistance?
If you want to do a simulation you would want to know that.
Alternately, what did you get for the output voltage?
Hint: I get a value for the output voltage that lies between 200 and 300 volts peak with an output resistance value that lies between 3000 and 3500 ohms. You should try to narrow this down by trying to calculate the required resistance and the output voltage.