I'm not a Maya user, but generally in other apps back-face culling just prevents the back-faces from being seen, which is why its normally found in the display options. If you're looking for something related to selections, enabling backface culling may only be a small piece of the puzzle. From other threads I see the…
Personally, I only use the option located in "Display> Polygons> Bacface Culling" and I UNcheck the "limit to selected" checkbox in the same menu. The camera based selection works very badly so I'm not using it.
In your example, this backface culling setting was applied to only the "selected" active selection ( if any ). That might lead to unexpected behavior if you simply meant to apply backface culling to "ALL" or on an object that wasn't selected. If that were true then the right answer would be to simply select the "all"…
I'm trying to only select the facing vertices. From what I've looked up it's called backculling and is accessed through the custom polygon display options. These are the options I set up, but it doesn't seem to deactivate the rear vertices. Anybody recognize what I'm doing wrong?