Is it ok to be a religious believer but to act outwardly atheist?

The short answer is found in in Matthew 10:32&33 "Whoever acknowledges me before men I will also acknowledge him before my Father in Heaven. But whoever disowns me before men, I will disown him before my Father in Heaven".