With the removal of "All iPhone Apps" link from the iTunes App Store, it's become harder to find the newest App Releases. To fill the void, readers can use AppShopper.com, a new site (of mine) that tracks New Apps, Updates and Price Changes for iPhone and iPod Touch applications.
The main view gives you a reverse chronological view of the latest changes found in the App Store. If you are interested specifically in just New Apps or just Price Changes, those can be easily filtered. Categories can also be listed individually.
A few useful examples:
http://appshopper.com/new - new apps
http://appshopper.com/prices - price drops and increases
http://appshopper.com/new/free - new free apps
http://appshopper.com/prices/free - apps that are now free
http://appshopper.com/games/new - all new games
Many additional features in the works (including search), but the core functionality is in place.