Automatic classification of sensitive content in remote sensing images, such as drug crop sites, is a promising task, as it can aid law-enforcement institutions in fighting illegal drug dealers worldwide, while, at the same time, it can help monitor legalized crops in countries that regulate them. However, existing art on detecting drug crops from remote sensing images is limited in some key factors, not taking full advantage of the available hyperspectral information for analysis. In this paper, departing from these methods, we propose a data-driven ensemble method to detect drug sites from remote sensing images. Our method comprises different convolutional neural network architectures applied to distinct image representations, which are able to represent complementary characterizations of such crops. To validate the proposed approach, we considered in our experiments a dataset containing Cannabis Sativa crops, spotted by police operations in a Brazilian region called the Marijuana Polygon. The results in this dataset show that our ensemble approach outperforms other data-driven and feature-engineering methods in a real-world experimental setup, in which unbalanced samples are present and acquisitions from different places in the same region are used for training and testing the methods, highlighting the promising use of this solution to aid police operations in detecting and collecting evidence of such sensitive content properly.