Varley was born at Hackney, then a village north of London, on 21 November 1781. He was a younger brother of John Varley, a watercolour painter and astrologer, and a close friend of William Blake. He was educated by his uncle, a scientific instrument maker, and under him acquired a knowledge of the natural sciences. Around 1800, he joined his brother in a tour through Wales, and began the study of art. He was soon engaged in teaching drawing. From 1803 to 1859 he was an occasional exhibitor at the Royal Academy and he also contributed regularly to the displays of the Water-Colour Society, of which, in 1803, he was one of the founders, and of which he continued a member until 1821. He died in Hampstead on 2 October 1873.
Varley’s artistic works consist mainly of carefully finished classical subjects, with architecture and figures. He published a series of etchings of boats and other craft on the River Thames. He was the first person to make a telephoto image.
His life as an artist was deeply connected to his scientific and technical pursuits. His optical inventions were improvements to the camera lucida and camera obscura long used by artists, and allowed him to draw both landscapes and microscopic studies in lucid detail. He also patented and sold his inventions.
The Society of Arts awarded him silver medals in 1831 and 1833, and the Isis gold medal in 1841, for improvements to microscopes. His innovations included lever-controlled stages and an illumination-modifier.
Varley invented the graphic telescope in 1809 and patented it in 1811. Unable to find a manufacturer who would make it, he went into manufacturing himself; this was his main occupation from 1814 onwards. He exhibited the graphic telescope at the Great Exhibition, winning a medal, in 1851. His firm also manufactured telegraphic equipment and testing apparatus.
He published Treatise on Optical Drawing Instruments in 1845.