Look at the following object. You'll notice that the object has rough edges. Artifacts like that is what people like to call jaggies. Lots of jaggies can be avoided when you use the anti-alias option (if available) of a particular tool as much as possible. In some situations jaggies still appear and I'm going to show you how to remove them. Let's use this object as an example.

Ctrl + left click (Command + left click on the Mac) on the layer that contains the object. The complete object will be selected (marching ants)

With the layer still active, add a mask. Click on the Add Layer Mask button(1) and a new mask (2) will appear.
Also notice that a new icon () appears in front of the layer (3) that reminds us that we have a layer mask and that the layer mask is active. A layer mask also has a double border if it's active and single one if it's not active.

Go in the menu to Filter/Blur/Gaussian Blur... and slightly blur our mask, just enough to see the jaggies disappear in the blur. In my case a value of 2.0 pixels was enough. This might be a larger or smaller value depending on the resolution of your image and the size of your jaggies.

Now comes the fun part. Go to Image/Adjust/Levels... and adjust the little sliders until the edge of the mask is sharp. In most cases you move the black slider close to the middle, with the white slider on the other side and the gray slider in between.

Important: Don't move all sliders too far to the right or you will make your object too small!