Well, the simple answer is that they are cheap and colorful, and that we like to watch things fly away.
Balloons in various forms were first invented for use in military communications, scientific experiments, and transportation, but soon enough, people began to have fun with them.
The first rubber balloons were made by Michael Faraday in 1824, for use in his laboratory. British inventor and rubber manufacturer Thomas Hancock began distributing balloon-making kits in the following year, and by 1847, London businesses were selling the first balloons made of vulcanized rubber, allowing for a stronger, more elastic balloon.
It was not that long before the idea had been exploited for use at parties.