Have you tried it according to the original recipe? If not, I recommend you do so.
If you have already done it and you found it too bitter,, put one bottle away for a few months and try it again later (it will mellow out, you'll see), and in the meantime, try it again and use just the juice and zest from the orange.
I have been breaking the warranty in this way for years because I found with my first batch that leaving the pith in gave me heartburn. I found the taste was just fine, the bitterness balances out all the sweetness, but since I'm good with sweetness, I like it pith-free as well, whereas other folks may find it too cloyingly sweet.