Alchemy mastery problem

Here is my problem.

I have taken as first part of quest for Alchemy Mastery the potion mastery specialization where i have to speak to somebody in Zangarmarsh. I have spoken to the master specialist there and accepted the quest but not done it yet even if i have the Botanica book for it plus nearly all potions needed to complete it.

IF i drop the quest, will it be possible for me to get back the choice of the three masteries specialization or it is finish and i will never be able to specialize?