Pastor Fired

I am not a member of any church nor do I have a desire to be one.  As an agnostic, I find it hard to fit into the hard fast mold of any church.  My wife and children, however, are members of a local church - conservative christian.   My interactions with this church overall, have been positive.  The people I have met for the most part seem to be genuine (any large group has its "eccentric" members), the staff seems to have the best interest of my family in mind (even if I don't agree with them theologically) and the pastor seems to be a genuinely nice, humble person.  

Out of the blue this past week, he was fired.  Effective immediately, no severance.  No reason was given, only a vague statement about the egregious nature of his "sin" and how his family will need time to heal.  I'm choosing to read "affair" into that.  It may not be and for the purposes of this little post, what he did really isn't important.  What concerns me is the shock that comes with his "failure".  This is big news!  People cried when the statement was read.  There's a bunch of whispering going on.  Until it's divulged what his offense was, I'm sure it will continue.  

This is my concern.  I liked the pastor.  He was a nice guy.  Really.  I'm not in the least bit shocked that this happened.  Not because I hate pastors or think they're all frauds (sometimes I wonder), but because he's just a guy.  If he cheated on his wife, I'm not surprised, but I am disappointed.  Not because he's a pastor, but because he let his emotions dictate his decision making process even when he knew they were going against a commitment he made to his wife.  Pastors are no different than lay people.  They've just spent more time studying a subject most people think is important but can't find much time to study themselves.  Standing up in front of people to discuss the findings of your studies doesn't make you a better person, it shouldn't hold you to a higher standard, it's just the job you're paid to do.  If I cheat on my wife, I'm an asshole, but I get to keep my job.  When a pastor does, he can no longer support his family and the reality of the situation is, unless you're a larger than life egomaniacal mega church pastor, you probably won't be pastoring a congregation any time soon.  

Why is this?  Why can't pastors be human too?  Why do they have to be held to a higher moral standard?