Work is slow and I feel guilty..?

I am self employed and work is slow and I get home wayyy to early (in my opinion) 5hr days are like half days. I feel guilty, my wife works, and yes I have employees. But if I lay them off, what happens when I get busy? Should I feel guilty? The dollars are minimal for paying everyone, but I run a tight ship, except for one guy who is basically a foreman (type) who gets a salary every week. So for him its good when we are slow..because he gets paid weather he works 20 or 40 hrs per week. He;s been on salary for over 5yrs and I dont know how to break him off it? Example, If we only work 5hrs one day, and 3 the next he would get 1 day salary and any off days he just will not get paid. Pay per day (Example- every day we work over 5 hrs you get your $100) for that day. I just feel I get screwed with salary now that things are slow- and I hate to let my hourly guys go. Any suggestions or comments.