Weed management is a key issue in agricultural systems. Standard (chemical) weed control is cost-effective but its impact on the environment is increasingly considered undesirable. Fortunately, help is on the way in the form of advances in robotics and computer vision. These advances make it possible to envision highly selective automated weed control where the use of herbicides is either greatly reduced, or replaced by non-chemical control methods. The current state-of-the-art of real-time weed detection and control is assessed by referring to our experiences in the contrasting cases of small vegetables, sugar beet and dairy pasture. We discuss the various ways to implement weed detection, weed control, and the way in which the level of confidence that a weed has been detected can be used to decide whether or not to apply the control. In general current detection techniques are insufficiently able to deal with the full range of conditions typically encountered in the field and increased robustness of weed detection systems is needed. This can be achieved by adopting a probabilistic framework that explicitly accounts for uncertainty in sensor readings and for uncertain information about the environment. We will give an overview of what might be involved in a probabilistic framework for weed detection.