If you don't want your software to change then don't upgrade.
If you want your software to improve then expect (and enjoy) change.
If a feature is present in a version X, then that feature may not exist in version X+1. This is not a regression - applications don't exist to provide features; they exist to serve the users that the application is targeted at.
Default applications are there to serve the greatest number of users. In both the proprietary and open-source worlds there is often a "basic" and an "advanced" application for a given task. This is a good thing - trying to please everyone isn't as good as focussing on one group.
If there is an application that better serves your need then use it. (If you are an advanced user then use an advanced application).
I'm personally most interested in writing applications for the beginner to intermediate user. I'll relate this to the applications that I develop:
- Simple Scan is not a replacement for xsane or gscan2pdf. But it is great if you have a picture you want to upload to Facebook.
- GCalctool does not have the features of Qalculate or wxMaxima. But it does solve common equations.
- glChess would not be used by a experienced chess player like xboard. But it is a good introduction to the game of Chess.