The compliance costs of private taxpayers are not only affected by the tax law itself but also by its implementation through the tax authorities. In the following paper we analyze the effect of administrative actions on the compliance costs of private businesses. We demonstrate in a theoretical model that compliance costs may partially be interpreted as externalities of authority behavior. As a result we expect a ”shifting” of administrative cost burdens from the tax administration to private taxpayers, what implies an economically inefficient outcome. Based on Belgian survey data, we find empirical evidence for the elucidated relationship. We give an quantitative estimate for the accordant effects and demonstrate, which activities of the administration are the most important cost drivers. Furthermore, we find empirical support that the effect of administrative issues is independent from the impact of the tax law itself.