The prior version of Xcode was offered for $4.99 on the App Store to non-developers when it was released earlier this year.
Apple has traditionally included updated developers tools for free on the OS install DVD. With the advent of digital delivery, Apple has chosen to offer Xcode as a separate app download.
It seems likely that the $4.99 charge was related to Apple's convoluted accounting practices related to the addition of features without receiving direct payment for them. Because an improved version of Xcode wasn't originally included with Snow Leopard, Apple was required to charge for it to comply with certain accounting rules.
In its earnings call yesterday, Apple noted a change in how it was accounting for a portion of Mac and OS X sales to avoid such problems in the future.