Why Does It Matter ?

In brief there are lots of reasons: moral, it's a challenge, legal / regulatory (e.g., section 508 in the U.S.), or because of the "cross-over" benefits of accessibility features (e.g., being able to have KDE speak to you in your car, for mobile users, etc.).

Section 508 in the U.S. is a regulation that mandates U.S. federal agencies have accessible IT. And the same is planned in the EU (Daniel Molkentin). More information on http://www.section508.gov/ (Bill Haneman).