For a healthy production line, it is essential to ensure a low failure rate of products. Product quality in production lines can be inspected using several techniques at the end of a production process, including a manual inspection. Different methods are applied to inspect the product quality at the end of the production process and sometimes during the production. This is often done using manual inspection, but this is less efficient, expensive, and time-consuming. Machine learning algorithms have the potential for evaluating and predicting product quality in a production line. In this paper, a novel product failure detection model that applies ANOVA (Analysis of Variance) feature selection method, Min-Max Scaling normalization method, mean imputation technique, Random Forest classification algorithm, a data sampling technique, and Grid Search parameter optimization approach is proposed and validated. For the comparison of the proposed model, several experiments have been performed using five classification algorithms, including RUSBoosted Tree. Experimental results demonstrated that the proposed model using the Random Forest algorithm, ANOVA feature selection, and sampling method achieves the best performance among other models and detects the faulty products effectively. It was also shown that the RUSBoosted Tree algorithm can be considered by practitioners for building the faulty product prediction model when data sampling and feature selection techniques are not integrated into the prediction model.