Do I use one hand or two hands to sign 'library'?
'Library' is a one-handed sign. You should use your dominant hand to form the 'L' shape and make the circular motion.
Does it matter which direction the circle goes?
Typically, the circle moves outward, down, inward, and back up. However, as long as you are making a clear, small circle with an 'L' handshape in front of your shoulder, you will be easily understood.
How big should the circular motion be?
The circle should be relatively small, about the size of a baseball or softball. Keep the movement comfortably contained in front of your shoulder area rather than making a wide circle.