A longstanding dream in software engineering research is to devise effective approaches for automating development tasks based on developers’ informally-specified intentions. Such intentions are generally in the form of natural language descriptions. In recent literature, a number of approaches have been proposed to automate tasks such as code search and even code generation based on natural language inputs. While these approaches vary in terms of technical designs, their objective is the same: transforming a developer’s intention into source code. The literature, however, lacks a comprehensive understanding towards the effectiveness of existing techniques as well as their complementarity to each other. We propose to fill this gap through a large-scale empirical study where we systematically evaluate natural language to code techniques. Specifically, we consider six state-of-the-art techniques targeting code search, and two targeting code generation. Through extensive evaluations on a dataset of 22K+ natural language queries, our study reveals the following major findings: (1) code search techniques based on model pre-training are so far the most effective while code generation techniques can also provide promising results\; (2) complementarities widely exist among the existing techniques\; and (3) combining the eight techniques together can gain an effectiveness enhancement of around 30% compared with the most effective standalone technique. Finally, we propose a strategy to automatically combine the results from different techniques based on their overlap degrees with the query. Experimental results show that our devised strategy is both effective and extensible.