Prior art on automated screening of diabetic retinopathy and direct referral decision shows promising performance; yet most methods build upon complex hand-crafted features whose performance often fails to generalize. Objective: We investigate data-driven approaches that extract powerful abstract representations directly from retinal images to provide a reliable referable diabetic retinopathy detector. Methods: We gradually build the solution based on convolutional neural networks, adding data augmentation, multi-resolution training, robust feature-extraction augmentation, and a patient-basis analysis, testing the effectiveness of each improvement. Results: The proposed method achieved an area under the ROC curve of 98.2% (95% CI: 97.4% – 98.9%) under a strict cross-dataset protocol designed to test the ability to generalize — training on the Kaggle competition dataset and testing using the Messidor-2 dataset. With a 5 × 2-fold cross-validation protocol, similar results are achieved for Messidor-2 and DR2 datasets, reducing the classification error by over 44% when compared to most published studies in existing literature. Conclusion: Additional boost strategies can improve performance substantially, but it is important to evaluate whether the additional (computation- and implementation-) complexity of each improvement is worth its benefits. We also corroborate that novel families of data-driven methods are the state of the art for diabetic retinopathy screening. Significance: By learning powerful discriminative patterns directly from available training retinal images, it is possible to perform referral diagnostics without detecting individual lesions.