The crux of meta-learning for few-shot learning is to design a meta-learner that effectively accrues knowledge during the learning process to improve the performance of a base-learner on new tasks with limited data. In this work, we introduce kernels with random Fourier features in the meta-learning framework to leverage their strong learning ability for few-shot learning. We propose meta variational random features (MetaVRF) to learn adaptive kernels for the base-learner, which is developed in a latent variable model by treating random feature basis as the latent variable. We formulate the optimization of MetaVRF as a variational inference problem by deriving an evidence lower bound under the meta-learning framework. To incorporate shared knowledge from related tasks, we propose a context inference of the posterior, which is established by an LSTM architecture. The LSTM-based inference network can effectively integrate the context information of previous tasks with task-specific information, which can generate informative and adaptive features. The learned MetaVRF can build up kernels of high representational power with a relatively low spectral sampling rate and also enables fast adaptation to new tasks. Experimental results on a variety of few-shot regression and classification tasks demonstrate that MetaVRF delivers much better, or at least competitive, performance compared to existing meta-learning alternatives.