With the rise of herbicide-resistant weeds, standard farming practices are no longer effective. Precision agriculture technologies can replace some of the standard practices but require information on the physical distribution of weed and crop species. In this work we outline a robotic weed management system and develop the image analysis portion of this system to test it with real aerial survey data. This portion of the system acts as an integral part of a robotic control loop, providing information on weed location and species to a robotic sprayer that will precisely apply the correct herbicide. The image analysis pipeline we develop proves the feasibility of obtaining this information from high altitude aerial surveys of agricultural land that already commonly take place. Our system performs well enough to reliably detect the presence of weed vegetation and to correctly classify it by species with an accuracy of 93.8 % between four classes of weed in images taken at an effective altitude of 80 feet (24.4 m).