Formation mechanisms are fundamental to the study of complex networks, but learning them from observations is challenging. In real-world domains, one often has access only to the final constructed graph, instead of the full construction process, and observed graphs exhibit complex, non-local structural properties. In this work, we propose GraphOpt, an end-to-end framework that jointly learns an implicit model of graph structure formation and discovers an underlying optimization mechanism in the form of a latent objective function. The learned objective can serve as an explanation for the observed graph properties, thereby lending itself to transfer across different graphs within a given domain. GraphOpt poses link formation in graphs as a sequential decision-making process and solves it using an efficient maximum entropy based inverse reinforcement learning algorithm. Further, it employs a novel continuous latent action space induced from node representations to promote scalability. We demonstrate empirically that GraphOpt discovers a latent objective and a robust stochastic policy that enable construction of graphs with properties similar to those in observed graph, transfer across graphs with different characteristics, and exhibit competitive performance on conventional downstream tasks such as link prediction, without being explicitly trained on these new graphs or task.