When working with layers, how do you only select items on that layer (prevent auto switching of layer in focus)?

Layers is a good feature. However I find it difficult to work with since all objects are clickable no matter what layer is selected. I would expect the system to allow me to interact with objects on the selected layer only.

CTRL + A should only select objects on the selected layer, not all objects on all layers.

Am I missing something? Is there a way to do this allready?