Masturbation is among the prohibited act. You should stop doing it even by having temporary marriage. The fast is valid if you did not know that masturbating is prohibited and that it will invalidate the fast.
The said act is permissible, but one should not annoy others as this may harm the reputation of the acts of commemorating the Ahlulbait (peace be upon them).