If looking for a physcially fit partner is important because that is your lifestyle, why not list it? Why is stating what you are looking for offensive to some? It's an attribute, like other things. Isn't that the whole point of the section "what you are looking for"? Some women should not jump to the conclusion it is all about physcial appearance. The more you let people know what you want, the less games you have to play.
While I don't state directly I want someone "physcially fit", I imply it by the activities I like to do. It does matter that they are nearly as physically fit as I am. I read their profile to see if they are physcially active, look at their pictures, and if there is any communication I ask about those types of things because it is important to me. It's not because I am superficial, but I want to be able to enjoy those activities with someone who isn't out of breath after a little bit of walking.
The reality is people are weedng others on this site for other things as well. How many hits do you get on your profile with no response? I get that almost all the time; views, but nothing beyond. It has to be something in my profile or the pictures they do not like or feel it is not a match. To endure on this site, you have to not take things too personally. People are looking for certain things in particular, and if that is includes "physcially fit", so be it.