To be able to select a single color with the Magic Wand tool, you need to use 3 different settings:

Eye Dropper tool  options1. Select the Eye Dropper tool and set the the Sample Size to Point Sample. This will not only have an effect on the Eye Dropper tool, but also on the Magic Wand tool.

2. Select the Magic Wand tool and set the Tolerance to 0.

Magic Wand options

3. Uncheck the Anti-Alias option.

 

 

Example 1
Example 1

Example 2
Example 1

Example 3
Example 1