This paper presents a novel method for generating a single polynomial approximation that produces correctly rounded results for all inputs of an elementary function for multiple representations. The generated polynomial approximation has the nice property that the first few lower degree terms produce correctly rounded results for specific representations of smaller bitwidths, which we call progressive performance. To generate such progressive polynomial approximations, we approximate the correctly rounded result and formulate the computation of correctly rounded polynomial approximations as a linear program inspired by the RLIBM project. In contrast to RLIBM, we avoid storing large lookup tables for the polynomial coefficients. We observe that the problem of computing polynomial approximations to elementary functions is a linear programming problem in low dimensions, i.e., with a small number of unknowns. We design a fast randomized algorithm for efficiently computing polynomial approximations with progressive performance. Our method produces polynomial approximations that are faster than the RLIBM project and other mainstream libraries while also having progressive performance.