i totally dont agree with suddenly putting your price up and changing your payment structure! especially with a long term client, all be it a bad one. The design industry is a small world and behaivior like that wount go down well if he then speaks to his other industry colleges
to be honest, the best way of sorting him out is for you to get serious with your emails and your business. tell him you will only allow for a certian amount of amendments for the price, charge him if he changes the design mid way through or chooses a different chair suddenly.
seriously, if hes late with payments, say 'your accountant has advised you, that a late payment passed the statutory 31 days will incure daily interest' as per your terms and conditions. if you dont have terms, write some! and include them in your fee proposal at the beginning of the project.
I hope you learn from this though for future clients. If you begin disorganized and don't have a structure in place, clients will take advantage both with your time and money...you will get more respect if you act professionally from the start and be open about what you charge and what the client should expect